Encoding method and device therefor, and decoding method and device therefor

ABSTRACT

Provided is a video decoding method including obtaining, from a bitstream, split information indicating whether a current block is to be split; when the split information does not indicate that the current block is to be split, decoding the current block according to encoding information about the current block; and when the split information indicates that the current block is to be split, splitting the current block into at least two lower blocks, obtaining encoding order information indicating an encoding order of the at least two lower blocks of the current block from the bitstream, determining a decoding order of the at least two lower blocks according to the encoding order information, and decoding the at least two lower blocks according to the decoding order.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation Application of U.S. patentapplication Ser. No. 16/345,950, filed on Apr. 29, 2019, which is aNational Stage of International Application No. PCT/KR2017/012135, filedOct. 31, 2017, claiming priority based on U.S. Patent Application No.62/415,619, filed Nov. 1, 2016, the contents of all of which areincorporated herein by reference in their entirety.

TECHNICAL FIELD

The present disclosure relates to a video encoding method and a videodecoding method, and more particularly, to an intra or inter predictionmethod for methods and devices for determining encoding and decodingorders of an image.

BACKGROUND ART

When a video of high quality is encoded, a large amount of data isrequired. However, because a bandwidth available for transmission of thevideo data is limited, a data rate applied to transmission of the videodata may be limited. Therefore, for efficient transmission of videodata, there is a need for video data encoding and decoding methods withminimal deterioration in image quality and increased compression rates.

Video data may be compressed by removing spatial redundancy and temporalredundancy between pixels. Because adjacent pixels generally have commoncharacteristics, encoding information of a data unit consisting ofpixels is transmitted to remove redundancy between the adjacent pixels.

Pixel values of the pixels included in the data unit are not directlytransmitted but information about a method of obtaining the pixel valuesis transmitted. A prediction method, in which a pixel value that issimilar to an original value is predicted, is determined for each dataunit, and encoding information about the prediction method istransmitted from an encoder to a decoder. Because a prediction value isnot completely equal to the original value, residual data of adifference between the original value and the prediction value istransmitted from the encoder to the decoder.

When prediction is exact, an amount of the encoding information forspecifying the prediction method is increased but a size of the residualdata is decreased. Therefore, the prediction method is determined, inconsideration of sizes of the encoding information and the residualdata. In particular, a data unit that is split from a picture hasvarious sizes, and in this regard, when a size of the data unit isincreased, there is an increased probability that accuracy of predictionis decreased, whereas an amount of encoding information is decreased.Thus, a size of a block is determined according to characteristics of apicture.

The prediction method includes intra prediction and inter prediction.The intra prediction is a method of predicting pixels of a block frompixels adjacent to the block. The inter prediction is a method ofpredicting pixels by referring to pixels of a different picture referredto for a picture including the block. Therefore, spatial redundancy isremoved by the intra prediction, and temporal redundancy is removed bythe inter prediction.

When the number of prediction methods is increased, an amount ofencoding information for indicating the prediction method is increased.Thus, the amount of the encoding information may be decreased bypredicting, from a different block, the encoding information to beapplied to a block.

Because loss of video data is allowed to the extent that the human eyecannot recognize the loss, residual data may be lossy-compressedaccording to transformation and quantization processes, and by doing so,an amount of the residual data may be decreased.

DESCRIPTION OF EMBODIMENTS Technical Problem

Provided is a video encoding method of determining whether to split acurrent block and an encoding order of lower blocks, and determining anencoding method according to whether neighboring blocks of the currentblock have been encoded. Provided is a video decoding method ofsplitting a current block, determining an encoding order of split lowerblocks, and determining an encoding method according to whetherneighboring blocks of the current block have been encoded. In addition,a computer-readable recording medium having recorded thereon a programfor executing the video encoding method and the video decoding methodaccording to an embodiment on a computer is provided.

Solution to Problem

Provided is a video decoding method including obtaining, from abitstream, split information indicating whether a current block is to besplit; when the split information does not indicate that the currentblock is to be split, decoding the current block according to encodinginformation about the current block; and when the split informationindicates that the current block is to be split, splitting the currentblock into at least two lower blocks, obtaining encoding orderinformation indicating an encoding order of the at least two lowerblocks of the current block from the bitstream, determining a decodingorder of the at least two lower blocks according to the encoding orderinformation, and decoding the at least two lower blocks according to thedecoding order.

Provided is a video decoding device including a block splitterconfigured to split a current block into at least two lower blocks whensplit information indicating whether the current block is to be splitindicates that the current block is to be split; an encoding orderdeterminer configured to determine, when the current block is split intothe at least two lower blocks, a decoding order of the at least twolower blocks according to encoding order information indicating anencoding order of the at least two lower blocks; a prediction methoddeterminer configured to determine a prediction method for the currentblock when the split information indicates that the current block is notto be split; and a decoder configured to reconstruct the current blockaccording to a result of prediction according to the prediction method.

Provided is a video encoding method including splitting a current blockinto at least two lower blocks; determining, according to a result ofthe splitting of the current block, whether to split the current block,and generating split information indicating whether the current block isto be split; according to coding efficiency of the current block,determining an encoding order of the at least two lower blocks of thecurrent block, and generating encoding order information indicating anencoding order of the at least two lower blocks; and outputting abitstream including the split information and the encoding orderinformation.

Provided is a video encoding device including an encoding informationgenerator configured to split a current block into at least two lowerblocks; determine, according to a result of the splitting of the currentblock, whether to split the current block; generate split informationindicating whether the current block is to be split, according to codingefficiency of the current block; determine an encoding order of the atleast two lower blocks of the current block; and generate encoding orderinformation indicating an encoding order of the at least two lowerblocks; and an output unit configured to output a bitstream includingthe split information and the encoding order information.

Provided is a non-transitory computer-readable recording medium havingrecorded thereon a program for performing the video encoding method andthe video decoding method.

The technical problems of the present disclosure are not limited to theaforementioned technical features, and other unstated technical problemsmay be inferred from embodiments below.

Advantageous Effects of Disclosure

Whether to split a current block and an encoding order of a lower blockare determined, and a prediction method for the lower block isdetermined according to the encoding order of the lower block, so thatcoding efficiency of an image is increased.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A illustrates a block diagram of an image encoding device based oncoding units according to a tree structure, according to an embodimentof the present disclosure.

FIG. 1B illustrates a block diagram of an image decoding device based oncoding units according to a tree structure, according to an embodiment.

FIG. 2 illustrates a process of determining at least one coding unitwhen a current coding unit is split, according to an embodiment.

FIG. 3 illustrates a process of determining at least one coding unitwhen a coding unit having a non-square shape is split, according to anembodiment.

FIG. 4 illustrates a process of splitting a coding unit based on atleast one of block shape information and split shape information,according to an embodiment.

FIG. 5 illustrates a method of determining a predetermined coding unitfrom among an odd number of coding units, according to an embodiment.

FIG. 6 illustrates an order in which a plurality of coding units areprocessed when a current coding unit is split and thus the plurality ofcoding units are determined, according to an embodiment.

FIG. 7 illustrates a process of determining a current coding unit to besplit into an odd number of coding units when coding units are unable tobe processed in a predetermined order, according to an embodiment.

FIG. 8 illustrates a process of determining at least one coding unitwhen a first coding unit is split, according to an embodiment.

FIG. 9 illustrates that, when a second coding unit having a non-squareshape, which is determined when a first coding unit is split, satisfiesa predetermined condition, a shape of the second coding unit that issplittable is limited, according to an embodiment.

FIG. 10 illustrates a process of splitting a coding unit having a squareshape when split shape information does not indicate that the codingunit is to be split into four coding units having square shapes,according to an embodiment.

FIG. 11 illustrates that a processing order between a plurality ofcoding units may be changed according to a split process of a codingunit, according to an embodiment.

FIG. 12 illustrates a process of determining a depth of a coding unitwhen a shape and size of the coding unit change, in a case where aplurality of coding units are determined when the coding unit isrecursively split, according to an embodiment.

FIG. 13 illustrates a depth determinable according to shapes and sizesof coding units, and a part index (PID) for distinguishing between thecoding units, according to an embodiment.

FIG. 14 illustrates that a plurality of coding units are determinedaccording to a plurality of predetermined data units included in apicture, according to an embodiment.

FIG. 15 illustrates a processing block that is a criterion indetermining a determining order of a reference coding unit included in apicture, according to an embodiment.

FIG. 16 illustrates a video decoding device involving splitting acurrent block and determining an encoding order of split lower blocks,according to an embodiment.

FIGS. 17A to 17C illustrate a default encoding order according to anembodiment.

FIGS. 18A and 18B respectively illustrate a case in which a coding unitis encoded in a forward direction and a case in which a coding unit isencoded in a backward direction.

FIG. 19 illustrates a tree structure of a largest coding unit fordescribing an encoding order of a largest coding unit and coding unitsincluded in the largest coding unit.

FIGS. 20A and 20B illustrate how an encoding order of at least threeblocks arranged in a vertical or horizontal direction is changedaccording to an encoding order flag.

FIG. 21 illustrates a method of determining a reference sample requiredin a directional intra prediction mode.

FIGS. 22A and 22B illustrate a prediction method in a discrete cosine(DC) mode according to whether a right block has been decoded.

FIGS. 23A to 23C illustrate a prediction method in a planar modeaccording to whether a right block has been decoded.

FIGS. 24A to 24D illustrate a method of predicting a current blockaccording to a multi-parameter intra (MPI) mode.

FIGS. 25A and 25B illustrate reference areas that are referred to in alinear-model (LM) chroma mode and a most probable chroma (MPC) mode.

FIG. 26 illustrates a block that is spatially adjacent to a currentblock according to an encoding order of the current block in a mergemode and an advanced motion vector prediction (AMVP) mode.

FIG. 27 illustrates a prediction method using a right block of a currentblock in an overlapped block motion compensation (OBMC) mode.

FIGS. 28A to 28C illustrate a prediction method using a right block of acurrent block in a sub-block motion vector prediction (MVP) mode.

FIGS. 29A and 29B illustrate a prediction method using a right block ofa current block in an affine motion compensation (MC) mode.

FIGS. 30A and 30B illustrate a prediction method using a right block ofa current block in a frame rate up conversion (FRUC) mode.

FIG. 31 illustrates a video decoding method according to an embodimentinvolving splitting a current block and determining an encoding order ofsplit lower blocks.

FIG. 32 illustrates a video encoding device according to an embodimentinvolving splitting a current block and determining an encoding order ofsplit lower blocks.

FIG. 33 illustrates a video encoding method according to an embodimentinvolving splitting a current block and determining an encoding order ofsplit lower blocks.

BEST MODE

Provided is a video decoding method including obtaining, from abitstream, split information indicating whether a current block is to besplit; when the split information does not indicate that the currentblock is to be split, decoding the current block according to encodinginformation about the current block; and when the split informationindicates that the current block is to be split, splitting the currentblock into at least two lower blocks, obtaining encoding orderinformation indicating an encoding order of the at least two lowerblocks of the current block from the bitstream, determining a decodingorder of the at least two lower blocks according to the encoding orderinformation, and decoding the at least two lower blocks according to thedecoding order.

Provided is a video decoding device including a block splitterconfigured to split a current block into at least two lower blocks whensplit information indicating whether the current block is to be splitindicates that the current block is to be split; an encoding orderdeterminer configured to determine, when the current block is split intothe at least two lower blocks, a decoding order of the at least twolower blocks according to encoding order information indicating anencoding order of the at least two lower blocks; a prediction methoddeterminer configured to determine a prediction method for the currentblock when the split information indicates that the current block is notto be split; and a decoder configured to reconstruct the current blockaccording to a result of prediction according to the prediction method.

Mode of Disclosure

Advantages and features of one or more embodiments of the presentdisclosure and methods of accomplishing the same may be understood morereadily by reference to the following detailed description of theembodiments and the accompanying drawings. In this regard, the presentdisclosure may, however, be embodied in many different forms and shouldnot be construed as being limited to the embodiments set forth herein.Rather, these embodiments are provided so that this disclosure will bethorough and complete and will fully convey the concept of the presentembodiments to one of ordinary skill in the art.

Hereinafter, the terms used in the specification will be brieflydefined, and the embodiments will be described in detail.

All terms including descriptive or technical terms which are used hereinshould be construed as having meanings that are obvious to one ofordinary skill in the art. However, the terms may have differentmeanings according to the intention of one of ordinary skill in the art,precedent cases, or the appearance of new technologies. Also, some termsmay be arbitrarily selected by the applicant, and in this case, themeaning of the selected terms will be described in detail in thedetailed description of the disclosure. Thus, the terms used herein haveto be defined based on the meaning of the terms together with thedescription throughout the specification.

An expression used in the singular encompasses the expression of theplural, unless it has a clearly different meaning in the context.

When a part “includes” or “comprises” an element, unless there is aparticular description contrary thereto, the part can further includeother elements, not excluding the other elements. Also, the term “unit”in the embodiments of the present disclosure means a software componentor hardware component such as a field-programmable gate array (FPGA) oran application-specific integrated circuit (ASIC), and performs specificfunctions. However, the term “unit” is not limited to software orhardware. The “unit” may be formed so as to be in an addressable storagemedium, or may be formed so as to operate one or more processors. Thus,for example, the term “unit” may refer to components such as softwarecomponents, object-oriented software components, class components, andtask components, and may include processes, functions, attributes,procedures, subroutines, segments of program code, drivers, firmware,micro codes, circuits, data, a database, data structures, tables,arrays, variables, or the like. A function provided by the componentsand “units” may be associated with the smaller number of components and“units”, or may be divided into additional components and “units”

The term “current block” refers to one of a coding unit, a predictionunit, and a transform unit which are currently to be encoded or decoded.In addition, the term “lower block” refers to a data unit split from the“current block”. The term “upper block” refers to a data unit includingthe “current block”.

Hereinafter, a “sample” is data allocated to a sampling location of animage and may mean data that is a processing target. For example, pixelvalues in an image of a spatial domain or transform coefficients on atransformation domain may be samples. A unit including at least onesample may be defined as a block.

The present disclosure will now be described more fully with referenceto the accompanying drawings for one of ordinary skill in the art to beable to perform the present disclosure without any difficulty. Inaddition, portions irrelevant to the descriptions of the presentdisclosure will be omitted in the drawings for clear descriptions of thepresent disclosure.

FIG. 1A illustrates a block diagram of an image encoding device 100based on coding units according to a tree structure, according to anembodiment of the present disclosure.

The image encoding device 100 includes a largest coding unit determiner110, a coding unit determiner 120, and an output unit 130.

The largest coding unit determiner 110 splits a picture or a sliceincluded in the picture into a plurality of largest coding units,according to a size of a largest coding unit. The largest coding unitmay be a data unit having a size of 32×32, 64×64, 128×128, 256×256, orthe like, wherein a shape of the data unit is a square having a widthand length in squares of 2. The largest coding unit determiner 110 mayprovide largest coding unit size information indicating the size of thelargest coding unit to the output unit 130. The largest coding unit sizeinformation may be included in a bitstream by the output unit 130.

The coding unit determiner 120 determines coding units by splitting thelargest coding unit. A coding unit may be determined by its largest sizeand depth. A depth may be defined as the number of times that the codingunit is spatially split from the largest coding unit. When the depth isincreased by 1, the coding unit is split into at least two coding units.Therefore, when the depth is increased, sizes of coding units accordingto depths are each decreased. Whether to split a coding unit isdetermined according to whether splitting the coding unit is efficientaccording to rate-distortion optimization. Then, split informationindicating whether the coding unit is to be split may be generated. Thesplit information may be expressed as a form of a flag.

The coding unit may be split by using various methods. For example, asquare coding unit may be split into four square coding units of whichwidth and height are half of those of the square coding unit. The squarecoding unit may be split into two rectangular coding units of whichwidth is half. The square coding unit may be split into two rectangularcoding units of which height is half. The square coding unit may besplit into three coding units in a manner that a width or height thereofis split by 1:2:1.

A rectangular coding unit of which width is twice as large as a heightmay be split into two square coding units. The rectangular coding unitof which width is twice as large as the height may be split into tworectangular coding units of which width is four times larger than aheight. The rectangular coding unit of which width is twice as large asthe height may be split into two rectangular coding units and one squarecoding unit in a manner that the width is split by 1:2:1.

Equally, a rectangular coding unit of which height is twice as large asa width may be split into two square coding units. The rectangularcoding unit of which height is twice as large as the width may be splitinto two rectangular coding units of which height is four times largerthan a width. Equally, the rectangular coding unit of which height istwice as large as the width may be split into two rectangular codingunits and one square coding unit in a manner that the height is split by1:2:1.

When the image encoding device 100 is capable of using two or more splitmethods, information about a split method that is usable to a codingunit, the split method being from among the split methods that areavailable to the image encoding device 100, may be determined for eachpicture. Therefore, only specific split methods may be used for eachpicture. When the image encoding device 100 uses only one split method,the information about a split method that is usable to a coding unit isnot separately determined.

When split information of a coding unit indicates that the coding unitis to be split, split shape information indicating a split method withrespect to the coding unit may be generated. When only one split methodis usable in a picture including the coding unit, the split shapeinformation may not be generated. When the split method is determined tobe adaptive to encoding information adjacent to the coding unit, thesplit shape information may not be generated.

The largest coding unit may be split to smallest coding units accordingto smallest coding unit size information. A depth of the largest codingunit may be defined to be an uppermost depth, and a depth of thesmallest coding units may be defined to be a lowermost depth. Therefore,a coding unit having an upper depth may include a plurality of codingunits having a lower depth.

As described above, according to a largest size of a coding unit, imagedata of a current picture is split into a largest coding unit. Thelargest coding unit may include coding units that are split according todepths. Because the largest coding unit is split according to thedepths, image data of a spatial domain included in the largest codingunit may be hierarchically split according to the depths.

A maximum depth that limits the maximum number of hierarchicallysplitting the largest coding unit or a minimum size of a coding unit maybe preset.

The coding unit determiner 120 compares coding efficiency ofhierarchically splitting a coding unit with coding efficiency of notsplitting the coding unit. Then, the coding unit determiner 120determines whether to split the coding unit according to a result of thecomparison. When the coding unit determiner 120 determines thatsplitting the coding unit is more efficient, the coding unit determiner120 hierarchically splits the coding unit. However, according to theresult of the comparison, when the coding unit determiner 120 determinesthat not splitting the coding unit is more efficient, the coding unitdeterminer 120 does not split the coding unit. Whether to split thecoding unit may be independently determined from whether a neighboringdifferent coding unit is split.

According to an embodiment, whether to split the coding unit may bedetermined from a coding unit having a large depth, during an encodingprocedure. For example, coding efficiency of a coding unit having amaximum depth is compared with coding efficiency of a coding unit havinga depth that is less than the maximum depth by 1, and it is determinedwhich one of coding units having the maximum depth and coding unitshaving the depth that is less than the maximum depth by 1 is efficientlyencoded in each area of a largest coding unit. According to a result ofthe determination, whether to split the coding units having the depththat is less than the maximum depth by 1 is determined in each area ofthe largest coding unit. Afterward, it is determined which one of codingunits having a depth that is less than the maximum depth by 2 and one ofthe coding units having the maximum depth and the coding units havingthe depth that is less than the maximum depth by 1, the one having beenselected according to the result of the determination, are furtherefficiently encoded in each area of the largest coding unit. The samedetermination process is performed on each of coding units having asmaller depth, and finally, whether to split the largest coding unit isdetermined according to which one of the largest coding unit and ahierarchical structure generated by hierarchically splitting the largestcoding unit is further efficiently encoded.

Whether to split the coding unit may be determined from a coding unithaving a small depth, during the encoding procedure. For example, codingefficiency of the largest coding unit is compared with coding efficiencyof a coding unit of which depth is greater than the largest coding unitby 1, and it is determined which one of the largest coding unit andcoding units of which depth is greater than the largest coding unit by 1is efficiently encoded. When the coding efficiency of the largest codingunit is better, the largest coding unit is not split. When codingefficiency of the coding units of which depth is greater than thelargest coding unit by 1 is better, the largest coding unit is split,and the comparison process is equally applied to split coding units.

When coding efficiency is examined from a coding unit having a largedepth, calculation is large but a tree structure having high codingefficiency is obtained. On the contrary, when the coding efficiency isexamined from a coding unit having a small depth, calculation is smallbut a tree structure having low coding efficiency is obtained.Therefore, in consideration of coding efficiency and calculation, analgorithm for obtaining a hierarchical tree structure of a largestcoding unit may be designed by using various methods.

To determine efficiency of a coding unit according to each depth, thecoding unit determiner 120 determines prediction and transformationmethods that are most efficient to the coding unit. To determine themost efficient prediction and transformation methods, the coding unitmay be split into predetermined data units. A data unit may have one ofvarious shapes according to a method of splitting the coding unit. Themethod of splitting the coding unit which is performed to determine thedata unit may be defined as a partition mode. For example, when a codingunit of 2N×2N (where N is a positive integer) is no longer split, a sizeof a prediction unit included in the coding unit is 2N×2N. When thecoding unit of 2N×2N is split, the size of the prediction unit includedin the coding unit may be 2N×N, N×2N, or N×N, according to the partitionmode. The partition mode according to the present embodiment maygenerate symmetrical data units obtained by symmetrically splitting aheight or width of the coding unit, data units obtained byasymmetrically splitting the height or width of the coding unit, such as1:n or n:1, data units obtained by diagonally splitting the coding unit,data units obtained by geometrically splitting the coding unit,partitions having arbitrary shapes, or the like.

The coding unit may be predicted and transformed based on a data unitincluded in the coding unit. However, according to the presentembodiment, a data unit for prediction and a data unit fortransformation may be separately determined. The data unit forprediction may be defined as a prediction unit, and the data unit fortransformation may be defined as a transform unit. A partition modeapplied to the prediction unit and a partition mode applied to thetransform unit may be different from each other, and prediction of theprediction unit and transformation of the transform unit may beperformed in a parallel and independent manner in the coding unit.

To determine an efficient prediction method, the coding unit may besplit into at least one prediction unit. Equally, to determine anefficient transformation method, the coding unit may be split into atleast one transform unit. The split into the prediction unit and thesplit into the transform unit may be independently performed from eachother. However, when a reconstructed sample in the coding unit is usedin intra prediction, a dependent relation is established betweenprediction units or transform units included in the coding unit, so thatthe split into the prediction unit and the transform unit may affecteach other.

The prediction unit included in the coding unit may be predicted throughintra prediction or inter prediction. The intra prediction involvespredicting prediction-unit samples by using reference samples adjacentto the prediction unit. The inter prediction involves predictingprediction-unit samples by obtaining reference samples from a referencepicture that is referred to for a current picture.

For the intra prediction, the coding unit determiner 120 may apply aplurality of intra prediction methods to the prediction unit, therebyselecting the most efficient intra prediction method. The intraprediction method includes a discrete cosine (DC) mode, a planar mode,directional modes such as a vertical mode and a horizontal mode, or thelike.

When a reconstructed sample adjacent to a coding unit is used as areference sample, the intra prediction may be performed on eachprediction unit. However, when a reconstructed sample in the coding unitis used as a reference sample, reconstruction with respect to thereference sample in the coding unit has to precede prediction withrespect to the reference sample in the coding unit, so that a predictionorder of a prediction unit may depend on a transformation order of atransform unit. Therefore, when the reconstructed sample in the codingunit is used as the reference sample, only an intra prediction methodfor transform units corresponding to the prediction unit, and actualintra prediction may be performed on each transform unit.

The coding unit determiner 120 may determine an optimal motion vectorand a reference picture, thereby selecting the most efficient interprediction method. For inter prediction, the coding unit determiner 120may determine a plurality of motion vector candidates from a coding unitthat is spatially and temporally adjacent to a current coding unit, andmay determine, from among them, the most efficient motion vector to be amotion vector. Equally, the coding unit determiner 120 may determine aplurality of reference picture candidates from the coding unit that isspatially and temporally adjacent to the current coding unit, and maydetermine the most efficient reference picture from among them.According to an embodiment, the reference picture may be determined fromreference picture lists that are predetermined with respect to a currentpicture. According to an embodiment, for accuracy of prediction, themost efficient motion vector from among the plurality of motion vectorcandidates may be determined to be a motion vector predictor, and amotion vector may be determined by compensating for the motion vectorpredictor. The inter prediction may be performed in parallel on eachprediction unit in the coding unit.

The coding unit determiner 120 may reconstruct the coding unit byobtaining only information indicating the motion vector and thereference picture, according to a skip mode. According to the skip mode,all encoding information including a residual signal is skipped, exceptfor the information indicating the motion vector and the referencepicture. Because the residual signal is skipped, the skip mode may beused when accuracy of prediction is very high.

A partition mode to be used may be limited according to the predictionmethod for the prediction unit. For example, only partition modes for aprediction unit having a size of 2N×2N or N×N may be applied to intraprediction, whereas partition modes for a prediction unit having a sizeof 2N×2N, 2N×N, N×2N, or N×N may be applied to inter prediction. Inaddition, only a partition mode for a prediction unit having a size of2N×2N may be applied to a skip mode of the inter prediction. The imageencoding device 100 may change a partition mode for each predictionmethod, according to coding efficiency.

The image encoding device 100 may perform transformation based on acoding unit or a transform unit included in the coding unit. The imageencoding device 100 may transform residual data that is a differencevalue between an original value and a prediction value with respect topixels included in the coding unit. For example, the image encodingdevice 100 may perform lossy-compression on the residual data throughquantization and discrete cosine transform (DCT)/discrete sine transform(DST). Alternatively, the image encoding device 100 may performlossless-compression on the residual data without the quantization.

The image encoding device 100 may determine a transform unit that is themost efficient one for quantization and transformation. The transformunit in the coding unit may be recursively split into smaller sizedregions in a manner similar to that in which the coding unit is splitaccording to the tree structure, according to an embodiment. Thus,residual data in the coding unit may be split according to the transformunit having the tree structure according to transformation depths. Theimage encoding device 100 may generate transformation split informationabout splitting the coding unit and the transform unit according to thedetermined tree structure of the transform unit.

A transformation depth indicating the number of splitting times to reachthe transform unit by splitting the height and width of the coding unitmay also be set in the image encoding device 100. For example, in acurrent coding unit of 2N×2N, a transformation depth may be 0 when thesize of a transform unit is 2N×2N, may be 1 when the size of thetransform unit is N×N, and may be 2 when the size of the transform unitis N/2×N/2. That is, the transform unit according to the tree structuremay be set according to the transformation depth.

In conclusion, the coding unit determiner 120 determines a predictionmethod that is the most efficient one for a current prediction unit andis from among a plurality of intra prediction methods and interprediction methods. Then, the coding unit determiner 120 determines aprediction unit determination scheme according to coding efficiencyaccording to a prediction result. Equally, the coding unit determiner120 determines a transform unit determination scheme according to codingefficiency according to a transformation result. According to the mostefficient prediction unit and transform unit determination scheme,coding efficiency of a coding unit is finally determined. The codingunit determiner 120 finalizes a hierarchical structure of a largestcoding unit, according to coding efficiency of a coding unit accordingto each depth.

The coding unit determiner 120 may measure coding efficiency of codingunits according to depths, prediction efficiency of prediction methods,or the like by using Rate-Distortion Optimization based on Lagrangianmultipliers.

The coding unit determiner 120 may generate split information indicatingwhether a coding unit is to be split according to each depth accordingto the determined hierarchical structure of the largest coding unit.Then, the coding unit determiner 120 may generate, for split codingunits, partition mode information to be used in determining a predictionunit and transform unit split information to be used in determining atransform unit. In addition, when the coding unit may be split by usingat least two split methods, the coding unit determiner 120 may generateboth split information and split shape information that indicates asplit method. The coding unit determiner 120 may generate informationabout the prediction method and the transformation method that are usedin the prediction unit and the transform unit.

The output unit 130 may output, in a bitstream, a plurality of pieces ofinformation generated by the largest coding unit determiner 110 and thecoding unit determiner 120 according to the hierarchical structure ofthe largest coding unit.

A method of determining the coding unit, the prediction unit, and thetransform unit according to the tree structure of the largest codingunit will be described below with reference to FIGS. 3 to 12 .

FIG. 1B illustrates a block diagram of an image decoding device 150based on coding units according to a tree structure, according to anembodiment.

The image decoding device 150 includes a receiver 160, an encodinginformation extractor 170, and a decoder 180.

Definitions of the terms including a coding unit, a depth, a predictionunit, a transform unit, various split information, or the like for adecoding operation performed by the image decoding device 150 are equalto those described above with reference to FIG. 1A and the imageencoding device 100. Because the image decoding device 150 is designedto reconstruct image data, various encoding methods used by the imageencoding device 100 may also be applied to the image decoding device150.

The receiver 160 receives and parses a bitstream with respect to anencoded video. The encoding information extractor 170 extracts, from theparsed bitstream, a plurality of pieces of information to be used indecoding largest coding units, and provides them to the decoder 180. Theencoding information extractor 170 may extract information about alargest size of a coding unit of a current picture from a header, asequence parameter set, or a picture parameter set of the currentpicture.

The encoding information extractor 170 extracts, from the parsedbitstream, a final depth and split information about coding unitsaccording to a tree structure according to each largest coding unit. Theextracted final depth and split information are output to the decoder180. The decoder 180 may split a largest coding unit according to theextracted final depth and split information, thereby determining a treestructure of the largest coding unit.

The split information extracted by the encoding information extractor170 is split information about the tree structure determined to cause aminimum encoding error, the determination being performed by the imageencoding device 100. Therefore, the image decoding device 150 mayreconstruct an image by decoding data according to a decoding schemethat causes the minimum encoding error.

The encoding information extractor 170 may extract split informationabout a data unit such as a prediction unit and a transform unitincluded in the coding unit. For example, the encoding informationextractor 170 may extract partition mode information about a partitionmode that is the most efficient one for the prediction unit. Theencoding information extractor 170 may extract transformation splitinformation about a tree structure that is the most efficient one forthe transform unit.

The encoding information extractor 170 may obtain information about themost efficient prediction method with respect to prediction units splitfrom the coding unit. Then, the encoding information extractor 170 mayobtain information about the most efficient transformation method withrespect to transform units split from the coding unit.

The encoding information extractor 170 extracts the information from thebitstream, according to a method of configuring the bitstream, themethod being performed by the output unit 130 of the image encodingdevice 100.

The decoder 180 may split a largest coding unit into coding units havingthe most efficient tree structure, based on the split information. Then,the decoder 180 may split the coding unit into the prediction unitsaccording to the partition mode information. The decoder 180 may splitthe coding unit into the transform units according to the transformationsplit information.

The decoder 180 may predict the prediction units according to theinformation about the prediction method. The decoder 180 may performinverse quantization and inverse transformation on residual data thatcorresponds to a difference between an original value and a predictionvalue of a pixel, according to information about a method oftransforming a transform unit. The decoder 180 may reconstruct pixels ofthe coding unit, according to a result of the prediction on theprediction units and a result of the transformation on the transformunits.

FIG. 2 illustrates a process of determining at least one coding unitwhen the image decoding device 150 splits a current coding unit,according to an embodiment.

According to an embodiment, the image decoding device 150 may determine,by using block shape information, a shape of a coding unit, and maydetermine, by using split shape information, a shape according to whichthe coding unit is to be split. That is, a method of splitting a codingunit, which is indicated by the split shape information, may bedetermined based on which block shape is indicated by the block shapeinformation used by the image decoding device 150.

According to an embodiment, the image decoding device 150 may use theblock shape information indicating that a current coding unit has asquare shape. For example, the image decoding device 150 may determinewhether to split a square coding unit or not, whether to split thesquare coding unit vertically, whether to split the square coding unithorizontally, or whether to split the square coding unit into fourcoding units, according to the split shape information. Referring toFIG. 2 , when block shape information of a current coding unit 200indicates a square shape, the decoder 180 may not split a coding unit210 a having the same size as the current coding unit 200 according tosplit shape information indicating no split, or may determine codingunits 210 b, 210 c, and 210 d split based on split shape informationindicating a predetermined split method.

Referring to FIG. 2 , the image decoding device 150 may determine thetwo coding units 210 b obtained by splitting the current coding unit 200in a vertical direction based on split shape information indicatingsplit in a vertical direction, according to an embodiment. The imagedecoding device 150 may determine the two coding units 210 c obtained bysplitting the current coding unit 200 in a horizontal direction based onsplit shape information indicating split in a horizontal direction. Theimage decoding device 150 may determine the four coding units 210 dobtained by splitting the current coding unit 200 in vertical andhorizontal directions based on split shape information indicating splitin vertical and horizontal directions. However, a split shape forsplitting a square coding unit may not be limitedly interpreted to aboveshapes, and may include various shapes indicatable by split shapeinformation. Predetermined split shapes for splitting a square codingunit will be described in detail below through various embodiments.

FIG. 3 illustrates a process of determining at least one coding unitwhen the image decoding device 150 splits a coding unit havingnon-square shape, according to an embodiment.

According to the present embodiment, the image decoding device 150 mayuse block shape information indicating that a current coding unit has anon-square shape. The image decoding device 150 may determine whether ornot to split the current coding unit having the non-square shape, orwhether to split the current coding unit having the non-square shape byusing a predetermined method. Referring to FIG. 3 , when block shapeinformation of a current coding unit 300 or 350 indicates a non-squareshape, the image decoding device 150 may not split a coding unit 310 or360 having the same size as the current coding unit 300 or 350 accordingto split shape information indicating no split, or may determine codingunits 320 a, 320 b, 330 a, 330 b, 330 c, 370 a, 370 b, 380 a, 380 b, and380 c split according to split shape information indicating apredetermined split method. A predetermined split method of splitting anon-square coding unit will be described in detail below through variousembodiments.

According to an embodiment, the image decoding device 150 may determine,by using the split shape information, a shape of a coding unit is split,and in this case, the split shape information may indicate the number ofat least one coding unit generated when a coding unit is split.Referring to FIG. 3 , when the split shape information indicates thatthe current coding unit 300 or 350 is to be split into two coding units,the image decoding device 150 may determine the two coding units 320 aand 320 b or 370 a and 370 b, which are respectively included in thecurrent coding unit 300 or 350 by splitting the current coding unit 300or 350 based on the split shape information.

According to an embodiment, when the image decoding device 150 splitsthe current coding unit 300 or 350 having the non-square shape based onthe split shape information, the image decoding device 150 may split thecurrent coding unit 300 or 350 having the non-square shape inconsideration of a location of a longer side. For example, the imagedecoding device 150 may determine a plurality of coding units bysplitting the current coding unit 300 or 350 in a direction of splittingthe longer sides of the current coding unit 300 or 350 in considerationof the shape of the current coding unit 300 or 350.

According to an embodiment, when split shape information indicates thata coding unit is to be split into an odd number of blocks, the imagedecoding device 150 may determine an odd number of coding units includedin the current coding unit 300 or 350. For example, when split shapeinformation indicates that the current coding unit 300 or 350 is to besplit into three coding units, the image decoding device 150 may splitthe current coding unit 300 or 350 into the three coding units 330 a,330 b, and 330 c or 380 a, 380 b, and 380 c. According to the presentembodiment, the image decoding device 150 may determine the odd numberof coding units included in the current coding unit 300 or 350, whereinsizes of the determined coding units are not the same. For example, asize of the coding unit 330 b or 380 b from among the odd number ofcoding units 330 a, 330 b, and 330 c or 380 a, 380 b, and 380 c may bedifferent from sizes of the coding units 330 a and 330 c or 380 a or 380c. That is, coding units that may be determined when the current codingunit 300 or 350 is split may have different types of sizes.

According to an embodiment, when split shape information indicates thata coding unit is to be split into an odd number of blocks, the imagedecoding device 150 may determine an odd number of coding units includedin the current coding unit 300 or 350 and in addition, set apredetermined limit on at least one coding unit from among the oddnumber of coding units generated by splitting the current coding unit300 or 350. Referring to FIG. 3 , the image decoding device 150 maydecode the coding unit 330 b or 380 b at the center of the three codingunits 330 a, 330 b, and 330 c or 380 a, 380 b, and 380 c generated whenthe current coding unit 300 or 350 is split in a different manner fromthe coding units 330 a and 330 c or 380 a and 380 c. For example, theimage decoding device 150 may limit the coding unit 330 b or 380 b atthe center not to be further split unlike the coding units 330 a and 330c or 380 a and 380 c, or to be split only a certain number of times.

FIG. 4 illustrates a process of splitting, by the image decoding device150, a coding unit based on at least one of block shape information andsplit shape information, according to an embodiment.

According to an embodiment, the image decoding device 150 may determinewhether to split a first coding unit 400 having a square shape intocoding units based on at least one of block shape information and splitshape information. According to an embodiment, when the split shapeinformation indicates a split of the first coding unit 400 in ahorizontal direction, the image decoding device 150 may determine asecond coding unit 410 by splitting the first coding unit 400 in thehorizontal direction. The terms “first coding unit”, “second codingunit”, and “third coding unit” according to an embodiment are used inthe context of splitting a coding unit. For example, a second codingunit may be determined when a first coding unit is split and a thirdcoding unit may be determined when the second coding unit is split.Relationships between the first through third coding units usedhereinafter may be understood to follow the above order characteristics.

According to an embodiment, the image decoding device 150 may determinewhether to split the determined second coding unit 410 into coding unitsbased on at least one of block shape information and split shapeinformation. Referring to FIG. 4 , the image decoding device 150 maysplit the second coding unit 410, which has a non-square shapedetermined by splitting the first coding unit 400, into at least onethird coding unit, for example, third coding units 420 a, 420 b, 420 c,and 420 d, based on at least one of block shape information and splitshape information, or may not split the second coding unit 410. Theimage decoding device 150 may obtain at least one of block shapeinformation and split shape information, the image decoding device 150may split the first coding unit 400 based on at least one of the blockshape information and the split shape information to obtain a pluralityof second coding units (for example, the second coding unit 410) havingvarious shapes, and the second coding unit 410 may be split according toa manner of splitting the first coding unit 400 based on at least one ofthe block shape information and the split shape information. Accordingto an embodiment, when the first coding unit 400 is split into thesecond coding units 410 based on at least one of block shape informationand split shape information about the first coding unit 400, the secondcoding unit 410 may also be split into the third coding units, forexample, the third coding units 420 a, 420 b, and 420 c, 420 d, based onat least one of block shape information and split shape informationabout the second coding unit 410. That is, a coding unit may berecursively split based on at least one of split shape information andblock shape information related to the coding unit. A method used torecursively split a coding unit will be described below through variousembodiments.

According to an embodiment, the image decoding device 150 may determineto split each of the third coding units (for example, the third codingunits 420 a, 420 b, 420 c, and 420 d) into coding units or not to splitthe second coding unit 410 based on at least one of block shapeinformation and split shape information. The image decoding device 150may split the second coding unit 410 having a non-square shape into theodd number of third coding units 420 b, 420 c, and 420 d. The imagedecoding device 150 may set a predetermined limitation on apredetermined third coding unit from among the odd number of thirdcoding units 420 b, 420 c, and 420 d. For example, the image decodingdevice 150 may limit the coding unit 420 c located at the center fromamong the odd number of third coding units 420 b, 420 c, and 420 d to besplit no more or to be split to a settable number of times. Referring toFIG. 4 , the image decoding device 150 may limit the coding unit 420 clocated at the center from among the odd number of third coding units420 b, 420 c, and 420 d included in the second coding unit 410 having anon-square shape to be split no more, to be split into a predeterminedsplit manner (for example, split only into four coding units or splitinto a shape corresponding to that into which the second coding unit 410is split), or to be split only a predetermined number of times (forexample, split only n times, wherein n>0). However, the limitations onthe coding unit 420 c located at the center are simply embodiments, andthus the present disclosure should not be interpreted limitedly to theabove embodiments, and it should be interpreted that the limitationsinclude various limitations of decoding the coding unit 420 c located atthe center differently from the coding units 420 b and 420 d.

According to an embodiment, the image decoding device 150 may obtain,from a predetermined location in a current coding unit, at least one ofblock shape information and split shape information used to split thecurrent coding unit.

According to an embodiment, when the current coding unit is split into apredetermined number of coding units, the image decoding device 150 mayselect one of the coding units. A method of selecting one of a pluralityof coding units may vary, and descriptions about such a method will bedescribed below through various embodiments.

According to an embodiment, the image decoding device 150 may split thecurrent coding unit into the plurality of coding units, and maydetermine the coding unit at the predetermined location.

FIG. 5 illustrates a method of determining, by the image decoding device150, a coding unit at a predetermined location from among an odd numberof coding units, according to an embodiment.

According to an embodiment, the image decoding device 150 may useinformation indicating a location of each of an odd number of codingunits so as to determine a coding unit located at the center of the oddnumber of coding units. Referring to FIG. 5 , the image decoding device150 may determine an odd number of coding units 520 a, 520 b, and 520 cby splitting a current coding unit 500. The image decoding device 150may determine the coding unit 520 b at the center by using informationabout locations of the odd number of coding units 520 a, 520 b, and 520c. For example, the image decoding device 150 may determine the codingunit 520 located at the center by determining locations of the codingunits 520 a, 520 b, and 520 c based on information indicating locationsof predetermined samples included in the coding units 520 a, 520 b, and520 c. In detail, the image decoding device 150 may determine the codingunit 520 b located at the center by determining the locations of thecoding units 520 a, 520 b, and 520 c based on information indicatinglocations of upper left samples 530 a, 530 b, and 530 c of the codingunits 520 a, 520 b, and 520 c.

According to an embodiment, the information indicating the locations ofthe upper left samples 530 a, 530 b, and 530 c respectively included inthe coding units 520 a, 520 b, and 520 c may include information aboutlocations or coordinates in a picture of the coding units 520 a, 520 b,and 520 c. According to an embodiment, the information indicating thelocations of the upper left samples 530 a, 530 b, and 530 c respectivelyincluded in the coding units 520 a, 520 b, and 520 c may includeinformation indicating widths or heights of the coding units 520 a, 520b, and 520 c included in the current coding unit 500, wherein the widthsor heights may correspond to information indicating differences betweencoordinates in the picture of the coding units 520 a, 520 b, and 520 c.That is, the image decoding device 150 may determine the coding unit 520b located at the center by directly using the information about thelocations or coordinates in the picture of the coding units 520 a, 520b, and 520 c, or by using the information about the widths or heights ofthe coding units, which indicate difference values between coordinates.

According to an embodiment, the information indicating the location ofthe upper left sample 530 a of the top coding unit 520 a may indicate(xa, ya) coordinates, information indicating the location of the upperleft sample 530 b of the center coding unit 520 b may indicate (xb, yb)coordinates, and the information indicating the location of the upperleft sample 530 c of the bottom coding unit 520 c may indicate (xc, yc)coordinates. The image decoding device 150 may determine the centercoding unit 520 b by using the coordinates of the upper left samples 530a, 530 b, and 530 c respectively included in the coding units 520 a, 520b, and 520 c. For example, when the coordinates of the upper leftsamples 530 a, 530 b, and 530 c are aligned in an ascending order ordescending order, the center coding unit 520 b including (xb, yb) thatis coordinates of the upper left sample 530 b may be determined as acoding unit located at the center from among the coding units 520 a, 520b, and 520 c determined when the current coding unit 500 is split. Here,the coordinates indicating the locations of the upper left samples 530a, 530 b, and 530 c may indicate coordinates indicating absolutelocations in the picture, and further, may use (dxb, dyb) coordinatesthat are information indicating a relative location of the upper leftsample 530 b of the center coding unit 520 b and (dxc, dyc) coordinatesthat are information indicating a relative location of the upper leftsample 530 c of the bottom coding unit 520 c, based on the location ofthe upper left sample 530 c of the top coding unit 520 a. Also, a methodof determining a coding unit at a predetermined location by usingcoordinates of a sample included in a coding unit as informationindicating a location of the sample should not be limitedly interpretedto the above method, and may be interpreted to various arithmeticmethods capable of using coordinates of a sample.

According to an embodiment, the image decoding device 150 may split thecurrent coding unit 500 into the plurality of coding units 520 a, 520 b,and 520 c, and select a coding unit from among the coding units 520 a,520 b, and 520 c according to a predetermined criterion. For example,the image decoding device 150 may select the coding unit 520 b that hasa different size from among the coding units 520 a, 520 b, and 520 c.

According to an embodiment, the image decoding device 150 may determinethe width or height of each of the coding units 520 a, 520 b, and 520 cby using the (xa, ya) coordinates that are the information indicatingthe location of the upper left sample 530 a of the top coding unit 520a, the (xb, yb) coordinates that are the information indicating thelocation of the upper left sample 530 b of the center coding unit 520 b,and the (xc, yc) coordinates that are the information indicating thelocation of the upper left sample 530 c of the bottom coding unit 520 c.The image decoding device 150 may determine a size of each of the codingunits 520 a, 520 b, and 520 c by using the coordinates (xa, ya), (xb,yb), and (xc, yc) indicating the locations of the coding units 520 a,520 b, and 520 c.

According to an embodiment, the image decoding device 150 may determinethe width of the top coding unit 520 a to xb-xa and the height to yb-ya.According to an embodiment, the image decoding device 150 may determinethe width of the center coding unit 520 b to xc-xb and the height toyc-yb. According to an embodiment, the image decoding device 150 maydetermine the width or height of the bottom coding unit by using thewidth or height of the current coding unit, and the width and height ofthe top coding unit 520 a and the center coding unit 520 b. The imagedecoding device 150 may determine one coding unit having a sizedifferent from other coding units based on the determined widths andheights of the coding units 520 a, 520 b, and 520 c. Referring to FIG. 5, the image decoding device 150 may determine, as the coding unit at thepredetermined location, the center coding unit 520 b having a sizedifferent from sizes of the top coding unit 520 a and the bottom codingunit 520 c. However, because a process of determining, by the imagedecoding device 150, a coding unit having a size different from othercoding units is only an embodiment of determining a coding unit at apredetermined location by using sizes of coding units determined basedon sample coordinates, various processes of determining a coding unit ata predetermined location by comparing sizes of coding units determinedaccording to predetermined sample coordinates may be used.

However, a location of a sample considered to determine a location of acoding unit should not be limitedly interpreted to the upper left, butmay be interpreted that information about a location of an arbitrarysample included in a coding unit is usable.

According to an embodiment, the image decoding device 150 may select acoding unit at a predetermined location from among an odd number ofcoding units that are determined when a current coding unit is split, inconsideration of a shape of the current coding unit. For example, whenthe current coding unit has a non-square shape in which a width islonger than a height, the image decoding device 150 may determine thecoding unit at the predetermined location along a horizontal direction.In other words, the image decoding device 150 may determine a codingunit from among coding units having different locations in thehorizontal direction, and may set a limitation on the coding unit. Whenthe current coding unit has the non-square shape in which the height islonger than the width, the image decoding device 150 may determine thecoding unit at the predetermined location along a vertical direction. Inother words, the image decoding device 150 may determine a coding unitfrom among coding units having different locations in the verticaldirection, and set a limitation on the coding unit.

According to an embodiment, the image decoding device 150 may useinformation indicating a location of each of an even number of codingunits so as to determine a coding unit at a predetermined location fromamong the even number of coding units. The image decoding device 150 maydetermine the even number of coding units by splitting a current codingunit, and determine the coding unit at the predetermined location byusing the information about the locations of the even number of codingunits. Detailed processes thereof may correspond to processes ofdetermining a coding unit at a predetermined location (for example, acenter location) from among an odd number of coding units, which havebeen described above with reference to FIG. 5 , and thus descriptionsthereof are not provided again.

According to an embodiment, when a current coding unit having anon-square shape is split into a plurality of coding units,predetermined information about a coding unit at a predeterminedlocation may be used during a split process so as to determine thecoding unit at the predetermined location from among the plurality ofcoding units. For example, the image decoding device 150 may use atleast one of block shape information and split shape information, whichare stored in a sample included in a center coding unit during a splitprocess so as to determine a coding unit located at the center fromamong a plurality of coding units obtained by splitting a current codingunit.

Referring to FIG. 5 , the image decoding device 150 may split thecurrent coding unit 500 into the plurality of coding units 520 a, 520 b,and 520 c based on at least one of block shape information and splitshape information, and determine the coding unit 520 b located at thecenter from among the plurality of coding units 520 a, 520 b, and 520 c.In addition, the image decoding device 150 may determine the coding unit520 b located at the center in consideration of a location where atleast one of the block shape information and the split shape informationis obtained. That is, at least one of the block shape information andthe split shape information of the current coding unit 500 may beobtained from the sample 540 located at the center of the current codingunit 500, and when the current coding unit 500 is split into theplurality of coding units 520 a, 520 b, and 520 c based on at least oneof the block shape information and the split shape information, thecoding unit 520 b including the sample 540 may be determined as thecoding unit located at the center. However, information used todetermine a coding unit located at the center should not be limitedlyinterpreted to at least one of block shape information and split shapeinformation, and various types of information may be used during aprocess of determining a coding unit located at the center.

According to an embodiment, predetermined information for identifying acoding unit at a predetermined location may be obtained from apredetermined sample included in a coding unit to be determined.Referring to FIG. 5 , the image decoding device 150 may use at least oneof block shape information and split shape information obtained from asample located at a predetermined location in the current coding unit500 (for example, a sample located at the center of the current codingunit 500) so as to determine a coding unit at a predetermined locationfrom among the plurality of coding units 520 a, 520 b, and 520 cdetermined when the current coding unit 500 is split (for example, acoding unit located at the center from among the plurality of codingunits). That is, the image decoding device 150 may determine the sampleat the predetermined location in consideration of a block shape of thecurrent coding unit 500, and the image decoding device 150 may determineand set a predetermined limitation on the coding unit 520 b includingthe sample from which predetermined location (for example, at least oneof the block shape information and the split shape information) isobtained, from among the plurality of coding units 520 a, 520 b, and 520c determined when the current coding unit 500 is split. Referring toFIG. 5 , the image decoding device 150 may determine the sample 540located at the center of the current coding unit 500, as the sample fromwhich the predetermined information is obtained, and the image decodingdevice 150 may set the predetermined location during a decoding process,on the coding unit 520 b including the sample 540. However, a locationof a sample from which predetermined information is obtained should notbe limitedly interpreted to the above location, and the sample may beinterpreted to samples at arbitrary locations included in the codingunit 520 determined to be limited.

According to an embodiment, a location of a sample from whichpredetermined location is obtained may be determined based on a shape ofthe current coding unit 500. According to an embodiment, block shapeinformation may be used to determine whether a shape of a current codingunit is a square or a non-square, and a location of a sample from whichpredetermined information is obtained may be determined based on theshape. For example, the image decoding device 150 may determine, as asample from which predetermined information is obtained, a samplelocated on a boundary of splitting at least one of a width and a heightof a current coding unit into halves by using at least one ofinformation about the width of the current coding unit and informationabout the height of the current coding unit. As another example, whenblock shape information about a current coding unit indicates anon-square shape, the image decoding device 150 may determine, as asample from which predetermined information is obtained, one of samplesadjacent to a boundary of splitting a longer side of the current codingunit into halves.

According to an embodiment, when a current coding unit is split into aplurality of coding units, the image decoding device 150 may use atleast one of block shape information and split shape information so asto determine a coding unit at a predetermined location from among theplurality of coding units. According to an embodiment, the imagedecoding device 150 may obtain at least one of the block shapeinformation and the split shape information from a sample at apredetermined location included in the coding unit, and the imagedecoding device 150 may split the plurality of coding units generatedwhen the current coding unit is split by using at least one of the splitshape information and the block shape information obtained from thesample at the predetermined location included in each of the pluralityof coding units. In other words, the coding unit may be recursivelysplit by using at least one of the block shape information and the splitshape information obtained from the sample at the predetermined locationin each coding unit. Because a recursive split process of a coding unithas been described above with reference to FIG. 4 , details thereof arenot provided again.

According to an embodiment, the image decoding device 150 may determineat least one coding unit by splitting a current coding unit, anddetermine an order of decoding the at least one coding unit according toa predetermined block (for example, a current coding unit).

FIG. 6 illustrates an order of processing a plurality of coding unitswhen the image decoding device 150 determines the plurality of codingunits by splitting a current coding unit, according to an embodiment.

According to an embodiment, the image decoding device 150 may determine,according to block shape information and split shape information, secondcoding units 610 a and 610 b by splitting a first coding unit 600 in avertical direction, second coding units 630 a and 630 b by splitting thefirst coding unit 600 in a horizontal direction, or second coding units650 a, 650 b, 650 c, and 650 d by splitting the first coding unit 600 invertical and horizontal directions.

Referring to FIG. 6 , the image decoding device 150 may determine anorder such that the second coding units 610 a and 610 b determined bysplitting the first coding unit 600 in the vertical direction to beprocessed in a horizontal direction 610 c. The image decoding device 150may determine a processing order of the second coding units 630 a and630 b determined by splitting the first coding unit 600 in thehorizontal direction to be in a vertical direction 630 c. The imagedecoding device 150 may determine the second coding units 650 a, 650 b,650 c, and 650 d determined by splitting the first coding unit 600 inthe vertical and horizontal directions to be processed according to apredetermined order (for example, a raster scan order or a z-scan order650 e) in which coding units in one row are processed and then codingunits in a next row are processed.

According to an embodiment, the image decoding device 150 mayrecursively split coding units. Referring to FIG. 6 , the image decodingdevice 150 may determine a plurality of coding units 610 a, 610 b, 630a, 630 b, 650 a, 650 b, 650 c, and 650 d by splitting the first codingunit 600, and may recursively split each of the determined plurality ofcoding units 610 a, 610 b, 630 a, 630 b, 650 a, 650 b, 650 c, and 650 d.A method of splitting the plurality of coding units 610 a, 610 b, 630 a,630 b, 650 a, 650 b, 650 c, and 650 d may be similar to a method ofsplitting the first coding unit 600. Accordingly, the plurality ofcoding units 610 a, 610 b, 630 a, 630 b, 650 a, 650 b, 650 c, and 650 dmay each be independently split into a plurality of coding units.Referring to FIG. 6 , the image decoding device 150 may determine thesecond coding units 610 a and 610 b by splitting the first coding unit600 in the vertical direction, and in addition, may determine to splitor not to split each of the second coding units 610 a and 610 bindependently.

According to an embodiment, the image decoding device 150 may split theleft second coding unit 610 a in the horizontal direction to obtainthird coding units 620 a and 620 b, and may not split the right secondcoding unit 610 b.

According to an embodiment, a processing order of coding units may bedetermined based on a split process of coding units. In other words, aprocessing order of split coding units may be determined based on aprocessing order of coding units just before being split. The imagedecoding device 150 may determine an order of processing the thirdcoding units 620 a and 620 b determined when the left second coding unit610 a is split independently from the right second coding unit 610 b.Because the third coding units 620 a and 620 b are determined when theleft second coding unit 610 a is split in the horizontal direction, thethird coding units 620 a and 620 b may be processed in a verticaldirection 620 c. Also, because the order of processing the left secondcoding unit 610 a and the right second coding unit 610 b is in thehorizontal direction 610 c, the third coding units 620 a and 620 bincluded in the left second coding unit 610 a may be processed in thevertical direction 620 c and then the right second coding unit 610 b maybe processed. Because the above descriptions are for describing aprocess of determining a processing order according to coding unitsbefore being split, the process should not be limitedly interpreted tothe above embodiments, and various methods of independently processingcoding units split and determined in various shapes according to apredetermined order may be used.

FIG. 7 illustrates a process of determining, by the image decodingdevice 150, a current coding unit to be split into an odd number ofcoding units when coding units are unable to be processed in apredetermined order, according to an embodiment.

According to an embodiment, the image decoding device 150 may determinethat the current coding unit is split into the odd number of codingunits based on obtained block shape information and split shapeinformation. Referring to FIG. 7 , a first coding unit 700 having asquare shape may be split into second coding units 710 a and 710 bhaving non-square shapes, and the second coding units 710 a and 710 bmay be independently split into third coding units 720 a, 720 b, 720 c,720 d, and 720 e. According to an embodiment, the image decoding device150 may determine a plurality of the third coding units 720 a and 720 bby splitting the left coding unit 710 a from among the second codingunits in a horizontal direction, and the right coding unit 710 b may besplit into an odd number of the third coding units 720 c, 720 d, and 720e.

According to an embodiment, the image decoding device 150 may determinewhether a coding unit split into an odd number exists by determiningwhether the third coding units 720 a, 720 b, 720 c, 720 d, and 720 e areprocessable in a predetermined order. Referring to FIG. 7 , the imagedecoding device 150 may determine the third coding units 720 a, 720 b,720 c, 720 d, and 720 e by recursively splitting the first coding unit700. The image decoding device 150 may determine, based on at least oneof block shape information and split shape information, whether there isa coding unit split into an odd number from among the first coding unit700, the second coding units 710 a and 710 b, and the third coding units720 a, 720 b, 720 c, 720 d, and 720 e. For example, a coding unitlocated at the right from among the second coding units 710 a and 710 bmay be split into the odd number of third coding units 720 c, 720 d, and720 e. An order of processing a plurality of coding units included inthe first coding unit 700 may be a predetermined order 730 (for example,a z-scan order), and the image decoding device 150 may determine whetherthe third coding units 720 c, 720 d, and 720 e determined when the rightsecond coding unit 710 b is split into an odd number satisfy a conditionof being processable according to the predetermined order.

According to an embodiment, the image decoding device 150 may determinewhether the third coding units 720 a, 720 b, 720 c, 720 d, and 720 eincluded in the first coding unit 700 satisfy a condition of beingprocessable according to a predetermined order, wherein the condition isrelated to whether at least one of a width and a height of the secondcoding units 710 a and 710 b is split into halves along boundaries ofthe third coding units 720 a, 720 b, 720 c, 720 d, and 720 e. Forexample, the third coding units 720 a and 720 b that are determined whenthe left second coding unit 710 a having a non-square shape is splitinto halves satisfy the condition, but the third coding units 720 c, 720d, and 720 e do not satisfy the condition because the boundaries of thethird coding units 720 c, 720 d, and 720 e that are determined when theright second coding unit 710 b is split into three coding units areunable to split a width or height of the right second coding unit 710 binto halves. Also, the image decoding device 150 may determinedisconnection of a scan order when the condition is dissatisfied, anddetermine that the right second coding unit 710 b is split into an oddnumber of coding units based on the determination result. According toan embodiment, when a coding unit is split into an odd number of codingunits, the image decoding device 150 may set a predetermined limitationon a coding unit at a predetermined location from among the codingunits, and because details about the limitation or the predeterminedlocation have been described above through various embodiments, detailsthereof are not provided again.

FIG. 8 illustrates a process of determining, by the image decodingdevice 150, at least one coding unit when a first coding unit 800 issplit, according to an embodiment. According to an embodiment, the imagedecoding device 150 may split the first coding unit 800 based on atleast one of block shape information and split shape informationobtained through the receiver 160. The first coding unit 800 having asquare shape may be split into four coding units having square shapes ornon-square shapes. For example, referring to FIG. 8 , when block shapeinformation indicates that the first coding unit 800 is a square andsplit shape information indicates that the first coding unit 800 is tobe split into non-square coding units, the image decoding device 150 maysplit the first coding unit 800 into a plurality of non-square codingunits. In detail, when the split shape information indicates that thefirst coding unit 800 is to be split into a horizontal or verticaldirection to determine an odd number of coding units, the image decodingdevice 150 may split the first coding unit 800 having a square shapeinto, as the odd number of coding units, second coding units 810 a, 810b, and 810 c determined when the first coding unit 800 is split in thevertical direction, or second coding units 820 a, 820 b, and 820 cdetermined when the first coding unit 800 is split in the horizontaldirection.

According to an embodiment, the image decoding device 150 may determinewhether the second coding units 810 a, 810 b, and 810 c and 820 a, 820b, and 820 c included in the first coding unit 800 satisfy a conditionof being processable according to a predetermined order, wherein thecondition is related to whether at least one of the width and the heightof the first coding unit 800 is split into halves along the boundariesof the second coding units 810 a, 810 b, and 810 c and 820 a, 820 b, and820 c. Referring to FIG. 8 , because the boundaries of the second codingunits 810 a, 810 b, and 810 c determined when the first coding unit 800having a square shape is split in the vertical direction are unable tosplit the width of the first coding unit 800 into halves, it may bedetermined that the first coding unit 800 does not satisfy the conditionof being processable according to the predetermined order. Also, becausethe boundaries of the second coding units 820 a, 820 b, and 820 cdetermined when the first coding unit 800 having a square shape is splitin the horizontal direction are unable to split the width of the firstcoding unit 800 into halves, it may be determined that the first codingunit 800 does not satisfy the condition of being processable accordingto the predetermined order. When the condition is dissatisfied, theimage decoding device 150 determines disconnection of a scan order andmay determine that the first coding unit 800 is split into an odd numberof coding units based on the determination result. According to anembodiment, when a coding unit is split into an odd number of codingunits, the image decoding device 150 may set a predetermined limitationon a coding unit at a predetermined location from among the codingunits, and because details about the limitation or the predeterminedlocation have been described above through various embodiments, detailsthereof are not provided again.

According to an embodiment, the image decoding device 150 may determinecoding units having various shapes by splitting a first coding unit.

Referring to FIG. 8 , the image decoding device 150 may split the firstcoding unit 800 having a square shape and a first coding unit 830 or 850having a non-square shape into coding units having various shapes.

FIG. 9 illustrates that, when a second coding unit having a non-squareshape, which is determined when a first coding unit 900 is split,satisfies a predetermined condition, a shape of the second coding unitthat is splittable is limited by the image decoding device 150,according to an embodiment.

According to an embodiment, the image decoding device 150 may determine,based on at least one of block shape information and split shapeinformation obtained through the receiver 160, to split the first codingunit 900 having a square shape into second coding units 910 a, 910 b,920 a, and 920 b having non-square shapes. The second coding units 910a, 910 b, 920 a, and 920 b may be independently split. Accordingly, theimage decoding device 150 may determine to split or not to split thesecond coding units 910 a, 910 b, 920 a, and 920 b based on at least oneof block shape information and split shape information related to eachof the second coding units 910 a, 910 b, 920 a, and 920 b. According toan embodiment, the image decoding device 150 may determine third codingunits 912 a and 912 b by splitting the left second coding unit 910 ahaving a non-square shape and determined when the first coding unit 900is split in a vertical direction. However, when the left second codingunit 910 a is split in a horizontal direction, the image decoding device150 may limit the right second coding unit 910 b not to be split in thehorizontal direction like a direction in which the left second codingunit 910 a is split. When the right second coding unit 910 b is split inthe same direction and third coding units 914 a and 914 b aredetermined, the third coding units 912 a, 912 b, 914 a, and 914 b may bedetermined when the left second coding unit 910 a and the right secondcoding unit 910 b are independently split in the horizontal direction.However, this is the same result as the image decoding device 150splitting the first coding unit 900 into four second coding units 930 a,930 b, 930 c, and 930 d having square shapes based on at least one ofblock shape information and split shape information, and thus may beinefficient in terms of image decoding.

According to an embodiment, the image decoding device 150 may determinethird coding units 922 a, 922 b, 924 a, and 924 b by splitting thesecond coding units 920 a or 920 b having a non-square shape anddetermined when the first coding unit 900 is split in the horizontaldirection. However, when one of second coding units (for example, thetop second coding unit 920 a) is split in the vertical direction, theimage decoding device 150 may limit the other second coding unit (forexample, the bottom second coding unit 920 b) not to be split in thevertical direction like a direction in which the top second coding unit920 a is split based on the above reasons.

FIG. 10 illustrates a process of splitting, by the image decoding device150, a coding unit having a square shape when split shape informationdoes not indicate that the coding unit is to be split into four codingunits having square shapes, according to an embodiment.

According to an embodiment, the image decoding device 150 may determinesecond coding units 1010 a, 1010 b, 1020 a, 1020 b, and the like bysplitting a first coding unit 1000 based on at least one of block shapeinformation and split shape information. The split shape information mayinclude information about various shapes into which a coding unit issplittable, but sometimes, the information about various shapes may notinclude information for splitting a coding unit into four square codingunits. According to such split shape information, the image decodingdevice 150 is unable to split the first coding unit 1000 having a squareshape into four square second coding units 1030 a, 1030 b, 1030 c, and1030 d. Based on the split shape information, the image decoding device150 may determine the second coding units 1010 a, 1010 b, 1020 a, 1020b, and the like having non-square shapes.

According to an embodiment, the image decoding device 150 mayindependently split the second coding units 1010 a, 1010 b, 1020 a, 1020b, and the like having non-square shapes. Each of the second codingunits 1010 a, 1010 b, 1020 a, 1020 b, and the like may be split in apredetermined order through a recursive method that may correspond to amethod of splitting the first coding unit 1000 based on at least one ofblock shape information and split shape information.

For example, the image decoding device 150 may determine third codingunits 1012 a and 1012 b having square shapes by splitting the leftsecond coding unit 1010 a in a horizontal direction and may determinethird coding units 1014 a and 1014 b having square shapes by splittingthe right second coding unit 1010 b in a horizontal direction. Inaddition, the image decoding device 150 may determine third coding units1016 a, 1016 b, 1016 c, and 1016 d having square shapes by splittingboth the left second coding unit 1010 a and the right second coding unit1010 b in the horizontal direction. In this case, coding units may bedetermined in the same manner in which the first coding unit 1000 issplit into the four square second coding units 1030 a, 1030 b, 1030 c,and 1030 d.

As another example, the image decoding device 150 may determine thirdcoding units 1022 a and 1022 b having square shapes by splitting the topsecond coding unit 1020 a in the vertical direction and determine thirdcoding units 1024 a and 1024 b having square shapes by splitting thebottom second coding unit 1020 b in the vertical direction. In addition,the image decoding device 150 may determine third coding units 1022 a,1022 b, 1024 a, and 1024 b having square shapes by splitting both thetop second coding unit 1020 a and the bottom second coding unit 1020 bin the vertical direction. In this case, coding units may be determinedin the same manner in which the first coding unit 1000 is split into thefour square second coding units 1030 a, 1030 b, 1030 c, and 1030 d.

FIG. 11 illustrates that a processing order between a plurality ofcoding units may be changed according to a split process of a codingunit, according to an embodiment.

According to an embodiment, the image decoding device 150 may split afirst coding unit 1100, based on block shape information and split shapeinformation. When the block shape information indicates a square shapeand the split shape information indicates that the first coding unit1100 is to be split in at least one of a horizontal direction and avertical direction, the image decoding device 150 may split the firstcoding unit 1100 to determine second coding units (for example, secondcoding units 1110 a, 1110 b, 1120 a, 1120 b, 1130 a, 1130 b, 1130 c,1130 d, and the like). Referring to FIG. 11 , the second coding units1110 a, 1110 b, 1120 a, and 1120 b having non-square shapes anddetermined when the first coding unit 1100 is split only in thehorizontal or vertical direction may each be independently split basedon block shape information and split shape information about each of thesecond coding units 1110 a, 1110 b, 1120 a, and 1120 b. For example, theimage decoding device 150 may determine third coding units 1116 a, 1116b, 1116 c, and 1116 d by splitting the second coding units 1110 a and1110 b in the horizontal direction, wherein the second coding units 1110a and 1110 b are generated when the first coding unit 1100 is split inthe vertical direction, and may determine third coding units 1126 a,1126 b, 1126 c, and 1126 d by splitting the second coding units 1120 aand 1120 b in the horizontal direction, wherein the second coding units1120 a and 1120 b are generated when the first coding unit 1100 is splitin the horizontal direction. Because split processes of the secondcoding units 1110 a, 1110 b, 1120 a, and 1120 b have been described withreference to FIG. 9 , details thereof are not provided again.

According to an embodiment, the image decoding device 150 may processcoding units according to a predetermined order. Because characteristicsabout processing of coding units according to a predetermined order havebeen described above with reference to FIG. 6 , details thereof are notprovided again. Referring to FIG. 11 , the image decoding device 150 maydetermine four square third coding units 1116 a, 1116 b, 1116 c, and1116 d or 1126 a, 1126 b, 1126 c, and 1126 d by splitting the firstcoding unit 1100 having a square shape. According to an embodiment, theimage decoding device 150 may determine a processing order of the thirdcoding units 1116 a, 1116 b, 1116 c, and 1116 d or 1126 a, 1126 b, 1126c, and 1126 d according to a shape of the first coding unit 1100 beingsplit.

According to an embodiment, the image decoding device 150 may determinethe third coding units 1116 a, 1116 b, 1116 c, and 1116 d by splittingeach of the second coding units 1110 a and 1110 b in the horizontaldirection, wherein the second coding units 1110 a and 1110 b aregenerated when the first coding unit 1100 is split in the verticaldirection, and the image decoding device 150 may process the thirdcoding units 1116 a, 1116 b, 1116 c, and 1116 d according to an order1117 of first processing the third coding units 1116 a and 1116 bincluded in the left second coding unit 1110 a in the vertical directionand then processing the third coding units 1116 c and 1116 d included inthe right second coding unit 1110 b in the vertical direction.

According to an embodiment, the image decoding device 150 may determinethe second coding units 1126 a, 1126 b, 1126 c, and 1126 d by splittingeach of the second coding units 1120 a and 1120 b in the verticaldirection, wherein the second coding units 1120 a and 1120 b aregenerated when the first coding unit 1100 is split in the horizontaldirection, and the image decoding device 150 may process the thirdcoding units 1126 a, 1126 b, 1126 c, and 1126 d according to an order offirst processing the third coding units 1126 a and 1126 b included inthe top second coding unit 1120 a in the horizontal direction and thenprocessing the third coding units 1126 c and 1126 d included in thebottom second coding unit 1120 b in the horizontal direction.

Referring to FIG. 11 , the third coding units 1116 a, 1116 b, 1116 c,1116 d, 1126 a, 1126 b, 1126 c, and 1126 d having square shapes may bedetermined when each of the second coding units 1110 a, 1110 b, 1120 a,and 1120 b are split. The second coding units 1110 a and 1110 bdetermined when the first coding unit 1100 is split in the verticaldirection and the second coding units 1120 a and 1120 b determined whenthe first coding unit 1100 is split in the horizontal direction havedifferent shapes, but according to the third coding units 1116 a, 1116b, 1116 c, 1116 d, 1126 a, 1126 b, 1126 c, and 1126 d determinedthereafter, the first coding unit 1100 is split into coding units havingthe same shapes. Accordingly, even when coding units having the sameshapes are determined as a result by recursively splitting coding unitsthrough different processes based on at least one of block shapeinformation and split shape information, the image decoding device 150may process the coding units having the same shapes in different orders.

FIG. 12 illustrates a process of determining a depth of a coding unitwhen a shape and size of the coding unit change, in a case where aplurality of coding units are determined when the coding unit isrecursively split, according to an embodiment.

According to an embodiment, the image decoding device 150 may determinea depth of a coding unit according to a predetermined criterion. Forexample, the predetermined criterion may be a length of a longer side ofthe coding unit. When a length of a longer side of a coding unit beforebeing split is 2n times a length of a longer side of a current codingunit, wherein n>0, the image decoding device 150 may determine that adepth of the current coding unit is higher than a depth of the codingunit before being split by n. Hereinafter, a coding unit having a higherdepth will be referred to as a coding unit of a lower depth.

Referring to FIG. 12 , according to an embodiment, the image decodingdevice 150 may determine a second coding unit 1202 and a third codingunit 1204 of lower depths by splitting a first coding unit 1200 having asquare shape, based on block shape information indicating a square shape(for example, block shape information may indicate ‘0: SQUARE’). When asize of the first coding unit 1200 having a square shape is 2N×2N, thesecond coding unit 1202 determined by splitting a width and a height ofthe first coding unit 1200 by ½ may have a size of N×N. In addition, thethird coding unit 1204 determined by splitting a width and a height ofthe second coding unit 1202 by ½ may have a size of N/2×N/2. In thiscase, a width and a height of the third coding unit 1204 correspond to ½times those of the first coding unit 1200. When a depth of the firstcoding unit 1200 is D, a depth of the second coding unit 1202, which is½ times the width and height of the first coding unit 1200, may be D+1,and a depth of the third coding unit 1204, which is ½ times the widthand height of the first coding unit 1200, may be D+2.

According to an embodiment, the image decoding device 150 may determinea second coding unit 1212 or 1222 and a third coding unit 1214 or 1224of lower depths by splitting a first coding unit 1210 or 1220 having anon-square shape, based on block shape information indicating anon-square shape (for example, the block shape information may indicate‘1: NS_VER’ indicating that a height is longer than a width or indicate‘2: NS_HOR’ indicating that a width is longer than a height). The imagedecoding device 150 may determine second coding units (for example, thesecond coding units 1202, 1212, 1222, and the like) by splitting atleast one of the width and the height of the first coding unit 1210having a size of N×2N. In other words, the image decoding device 150 maydetermine the second coding unit 1202 having a size of N×N or the secondcoding unit 1222 having a size of N×N/2 by splitting the first codingunit 1210 in a horizontal direction, or may determine the second codingunit 1212 having a size of N/2×N by splitting the first coding unit 1210in horizontal and vertical directions.

According to an embodiment, the image decoding device 150 may determinethe second coding units (for example, the second coding units 1202,1212, 1222, and the like) by splitting at least one of the width and theheight of the first coding unit 1220 having a size of 2N×N. That is, theimage decoding device 150 may determine the second coding unit 1202having a size of N×N or the second coding unit 1212 having a size ofN/2×N by splitting the first coding unit 1220 in the vertical direction,or may determine the second coding unit 1222 having a size of N×N/2 bysplitting the first coding unit 1220 in the horizontal and verticaldirections. According to an embodiment, the image decoding device 150may determine third coding units (for example, the third coding units1204, 1214, 1224, and the like) by splitting at least one of a width anda height of the second coding unit 1202 having a size of N×N. That is,the image decoding device 150 may determine the third coding unit 1204having a size of N/2×N/2, the third coding unit 1214 having a size ofN/2×N/2, or the third coding unit 1224 having a size of N/2×N/2 bysplitting the second coding unit 1202 in vertical and horizontaldirections.

According to an embodiment, the image decoding device 150 may determinethe third coding units (for example, the third coding units 1204, 1214,1224, and the like) by splitting at least one of a width and a height ofthe second coding unit 1212 having a size of N/2×N. That is, the imagedecoding device 150 may determine the third coding unit 1204 having asize of N/2×N/2 or the third coding unit 1224 having a size of N/2×N/2by splitting the second coding unit 1212 in a horizontal direction, ordetermine the third coding unit 1214 having a size of N/2×N/2 bysplitting the second coding unit 1212 in vertical and horizontaldirections.

According to an embodiment, the image decoding device 150 may determinethe third coding units (for example, the third coding units 1204, 1214,1224, and the like) by splitting at least one of a width and a height ofthe second coding unit 1214 having a size of N×N/2. That is, the imagedecoding device 150 may determine the third coding unit 1204 having asize of N/2×N/2 or the third coding unit 1214 having a size of N/2×N/2by splitting the second coding unit 1212 in a vertical direction, ordetermine the third coding unit 1224 having a size of N/2×N/2 bysplitting the second coding unit 1212 in vertical and horizontaldirections.

According to an embodiment, the image decoding device 150 may splitcoding units having square shapes (for example, the first coding units1200, 1202, and 1204) in a horizontal or vertical direction. Forexample, the first coding unit 1200 having a size of 2N×2N may be splitin the vertical direction to determine the first coding unit 1210 havinga size of N×2N or in the horizontal direction to determine the firstcoding unit 1220 having a size of 2N×N/. According to an embodiment,when a depth is determined based on a length of a longest side of acoding unit, a depth of a coding unit determined when the first codingunit 1200, 1202, or 1204 is split in the horizontal or verticaldirection may be the same as a depth of the first coding unit 1200,1202, or 1204.

According to an embodiment, the width and height of the third codingunit 1214 or 1224 may be ½ times the first coding unit 1210 or 1220.When the depth of the first coding unit 1210 or 1220 is D, the depth ofthe second coding unit 1212 or 1214, which is ½ times the width andheight of the first coding unit 1210 or 1220, may be D+1, and the depthof the third coding unit 1214 or 1224, which is ½ times the width andheight of the first coding unit 1210 or 1220, may be D+2.

FIG. 13 illustrates a depth determinable according to shapes and sizesof coding units, and a part index (PID) for distinguishing between thecoding units, according to an embodiment.

According to an embodiment, the image decoding device 150 may determinesecond coding units having various shapes by splitting a first codingunit 1300 having a square shape. Referring to FIG. 13 , the imagedecoding device 150 may determine second coding units 1302 a, 1302 b,1304 a, 1304 b, 1306 a, 1306 b, 1306 c, and 1306 d by splitting thefirst coding unit 1300 in at least one of a vertical direction and ahorizontal direction, according to split shape information. That is, theimage decoding device 150 may determine the second coding units 1302 a,1302 b, 1304 a, 1304 b, 1306 a, 1306 b, 1306 c, and 1306 d based onsplit shape information about the first coding unit 1300.

According to an embodiment, depths of the second coding units 1302 a,1302 b, 1304 a, 1304 b, 1306 a, 1306 b, 1306 c, and 1306 d determinedaccording to the split shape information about the first coding unit1300 having a square shape may be determined based on lengths of longersides. For example, because lengths of longer sides of the second codingunits 1302 a, 1302 b, 1304 a, and 1304 b having non-square shapes arethe same as a length of one side of the first coding unit 1300 having asquare shape, depths of the first coding unit 1300 and the second codingunits 1302 a, 1302 b, 1304 a, and 1304 b having non-square shapes may beD, i.e., the same. On the other hand, when the image decoding device 150splits the first coding unit 1300 into the four second coding units 1306a, 1306 b, 1306 c, and 1306 d having square shapes based on split shapeinformation, because a length of one side of each of the second codingunits 1306 a, 1306 b, 1306 c, and 1306 d having square shapes is ½ of alength of one side of the first coding unit 1300, depths of the secondcoding units 1306 a, 1306 b, 1306 c, and 1306 d may be D+1, i.e., onedepth lower than the depth D of the first coding unit 1300.

According to an embodiment, the image decoding device 150 may split afirst coding unit 1310 having a height longer than a width into aplurality of second coding units 1312 a, 1312 b, 1314 a, 1314 b, and1314 c by splitting the first coding unit 1310 in a horizontal directionaccording to split shape information. According to an embodiment, theimage decoding device 150 may split a first coding unit 1320 having awidth longer than a height into a plurality of second coding units 1322a and 1322 b, or 1324 a, 1324 b, and 1324 c by splitting the firstcoding unit 1320 in a vertical direction according to split shapeinformation.

According to an embodiment, depths of the second coding units 1312 a,1312 b, 1314 a, 1314 b, 1316 a, 1316 b, 1316 c, and 1316 d determinedaccording to the split shape information about the first coding unit1310 or 1320 having a non-square shape may be determined based onlengths of longer sides. For example, because a length of one side ofeach of the second coding units 1312 a and 1312 b having square shapesis ½ of a length of one side of the first coding unit 1310 having anon-square shape in which a height is longer than a width, the depths ofthe second coding units 1302 a, 1302 b, 1304 a, and 1304 b having squareshapes are D+1, i.e., one depth lower than the depth D of the firstcoding unit 1310 having a non-square shape.

In addition, the image decoding device 150 may split the first codingunit 1310 having a non-square shape into an odd number of the secondcoding units 1314 a, 1314 b, and 1314 c based on split shapeinformation. The odd number of second coding units 1314 a, 1314 b, and1314 c may include the second coding units 1314 a and 1314 c havingnon-square shapes and the second coding unit 1314 b having a squareshape. Here, because lengths of longer sides of the second coding units1314 a and 1314 c having non-square shapes and a length of one side ofthe second coding unit 1314 b having a square shape are ½ of a length ofone side of the first coding unit 1310, depths of the second codingunits 1314 a, 1314 b, and 1314 c may be D+1, i.e., one depth lower thanthe depth D of the first coding unit 1310. The image decoding device 150may determine depths of coding units related to the first coding unit1310 having a non-square shape in which a width is longer than a heightin the similar manner as depths of coding units related to the firstcoding unit 1310 are determined.

According to an embodiment, while determining PIDs for distinguishingbetween coding units, the image decoding device 150 may determine thePIDs based on size ratios between the coding units when an odd number ofthe coding units do not have the same size. Referring to FIG. 13 , thecoding unit 1314 b located at the center of the odd number of codingunits 1314 a, 1314 b, and 1314 c has the same width as the coding units1314 a and 1314 c, but has a height twice higher than heights of thecoding units 1314 a and 1314 c. In this case, the coding unit 1314 blocated at the center may include two of each of the coding units 1314 aand 1314 c. Accordingly, when a PID of the coding unit 1314 b located atthe center according to a scan order is 1, a PID of the coding unit 1314c located in a next order may be increased by 2, i.e., 3. That is,values of PIDs may be discontinuous. According to the presentembodiment, the image decoding device 150 may determine whether an oddnumber of coding units have the same size based on discontinuity of PIDsfor distinguishing between the coding units.

According to an embodiment, the image decoding device 150 may determinewhether a plurality of coding units determined when a current codingunit is split have certain split shapes based on values of PIDs fordistinguishing between the coding units. Referring to FIG. 13 , theimage decoding device 150 may determine an even number of the codingunits 1312 a and 1312 b or an odd number of the coding units 1314 a,1314 b, and 1314 c by splitting the first coding unit 1310 having arectangular shape in which a height is longer than a width. The imagedecoding device 150 may use an ID indicating each coding unit so as todistinguish between a plurality of coding units. According to anembodiment, the PID may be obtained from a sample at a predeterminedlocation (for example, an upper left sample) of each coding unit.

According to an embodiment, the image decoding device 150 may determinea coding unit at a predetermined location from among coding unitsdetermined via split, by using PIDs for distinguishing between thecoding units. According to an embodiment, when split shape informationabout the first coding unit 1310 having a rectangular shape in which aheight is longer than a width indicates a split into three coding units,the image decoding device 150 may split the first coding unit 1310 intothe three coding units 1314 a, 1314 b, and 1314 c. The image decodingdevice 150 may allocate a PID to each of the three coding units 1314 a,1314 b, and 1314 c. The image decoding device 150 may compare PIDs ofcoding units so as to determine a center coding unit from among an oddnumber of coding units. The image decoding device 150 may determine thecoding unit 1314 b having a PID corresponding to a center value fromamong PIDs as a coding unit located at the center from among codingunits determined when the first coding unit 1310 is split, based on PIDsof the coding units. According to an embodiment, the image decodingdevice 150 may determine PIDs based on size ratios between coding unitswhen the coding units do not have the same size, while determining thePIDs for distinguishing between the coding units. Referring to FIG. 13 ,the coding unit 1314 b generated when the first coding unit 1310 issplit may have the same width as the coding units 1314 a and 1314 c, butmay have a height twice higher than heights of the coding units 1314 aand 1314 c. In this case, when the PID of the coding unit 1314 b locatedat the center is 1, the PID of the coding unit 1314 c located in a nextorder may be increased by 2, i.e., 3. As such, when an increase rangechanges while PIDs are uniformly increasing, the image decoding device150 may determine that a coding unit is split into a plurality of codingunits including a coding unit having a different size from other codingunits. According to an embodiment, when split shape informationindicates a split into an odd number of coding units, the image decodingdevice 150 may split a current coding unit into an odd number of codingunits in which a coding unit at a predetermined location (for example, acenter coding unit) has a different size from other coding units. Inthis case, the image decoding device 150 may determine the center codingunit having the different size by using PIDs of the coding units.However, because the PID, and a size or location of a coding unit at apredetermined location are specified to describe an embodiment, and thusthe present disclosure is not limited thereto, and various PIDs, andvarious locations and sizes of a coding unit may be used.

According to an embodiment, the image decoding device 150 may use apredetermined data unit from which a coding unit starts to berecursively split. FIG. 14 illustrates that a plurality of coding unitsare determined according to a plurality of predetermined data unitsincluded in a picture, according to an embodiment.

According to an embodiment, a predetermined data unit may be defined asa data unit from which a coding unit starts to be recursively split byusing at least one of block shape information and split shapeinformation. That is, the predetermined data unit may correspond to acoding unit of an uppermost depth used while determining a plurality ofcoding units splitting a current picture. Hereinafter, for convenienceof description, such a predetermined data unit is referred to as areference data unit.

According to an embodiment, a reference data unit may indicate apredetermined size and shape. According to an embodiment, a referencecoding unit may include M×N samples. Here, M and N may be equal to eachother, and may be an integer expressed as a multiple of 2. That is, areference data unit may indicate a square shape or a non-square shape,and may later be split into an integer number of coding units.

According to an embodiment, the image decoding device 150 may split acurrent picture into a plurality of reference data units. According toan embodiment, the image decoding device 150 may split the plurality ofreference data units obtained by splitting the current picture by usingsplit information about each of the reference data units. Splitprocesses of such reference data units may correspond to split processesusing a quad-tree structure.

According to an embodiment, the image decoding device 150 maypre-determine a smallest size available for the reference data unitincluded in the current picture. Accordingly, the image decoding device150 may determine the reference data unit having various sizes that areequal to or larger than the smallest size, and determine at least onecoding unit based on the determined reference data unit by using blockshape information and split shape information.

Referring to FIG. 14 , the image decoding device 150 may use a referencecoding unit 1400 having a square shape, or may use a reference codingunit 1402 having a non-square shape. According to an embodiment, a shapeand size of a reference coding unit may be determined according tovarious data units (for example, a sequence, a picture, a slice, a slicesegment, and a largest coding unit) that may include at least onereference coding unit.

According to an embodiment, the receiver 160 of the image decodingdevice 150 may obtain, from a bitstream, at least one of informationabout a shape of a reference coding unit and information about a size ofthe reference coding unit, according to the various data units.Processes of determining at least one coding unit included in thereference coding unit 1400 having a square shape have been describedabove through processes of splitting the current coding unit 1000 ofFIG. 10 , and processes of determining at least one coding unit includedin the reference coding unit 1402 having a non-square shape have beendescribed above through processes of splitting the current coding unit1100 or 1150 of FIG. 11 , and thus descriptions thereof are not providedhere.

According to an embodiment, to determine a size and shape of a referencecoding unit according to some data units pre-determined based on apredetermined condition, the image decoding device 150 may use a PID forchecking the size and shape of the reference coding unit. That is, thereceiver 160 may obtain, from a bitstream, only a PID for checking asize and shape of a reference coding unit as a data unit satisfying apredetermined condition (for example, a data unit having a size equal toor smaller than a slice) from among various data units (for example, asequence, a picture, a slice, a slice segment, and a largest codingunit), according to slices, slice segments, and largest coding units.The image decoding device 150 may determine the size and shape of thereference data unit according to data units that satisfy thepredetermined condition, by using the PID. When information about ashape of a reference coding unit and information about a size of areference coding unit are obtained from a bitstream and used accordingto data units having relatively small sizes, usage efficiency of thebitstream may not be sufficient, and thus instead of directly obtainingthe information about the shape of the reference coding unit and theinformation about the size of the reference coding unit, only a PID maybe obtained and used. In this case, at least one of the size and theshape of the reference coding unit corresponding to the PID indicatingthe size and shape of the reference coding unit may be pre-determined.That is, the image decoding device 150 may select at least one of thepre-determined size and shape of the reference coding unit according tothe PID so as to determine at least one of the size and shape of thereference coding unit included in a data unit that is a criterion forobtaining the PID.

According to an embodiment, the image decoding device 150 may use atleast one reference coding unit included in one largest coding unit.That is, a largest coding unit splitting an image may include at leastone reference coding unit, and a coding unit may be determined when eachof the reference coding unit is recursively split. According to anembodiment, at least one of a width and height of the largest codingunit may be an integer times at least one of a width and height of thereference coding unit. According to an embodiment, a size of a referencecoding unit may be equal to a size of a largest coding unit, which issplit n times according to a quad-tree structure. That is, the imagedecoding device 150 may determine a reference coding unit by splitting alargest coding unit n times according to a quad-tree structure, andsplit the reference coding unit based on at least one of block shapeinformation and split shape information according to variousembodiments.

FIG. 15 illustrates a processing block that is a criterion indetermining a determining order of a reference coding unit included in apicture 1500, according to an embodiment.

According to an embodiment, the image decoding device 150 may determineat least one processing block splitting a picture. A processing block isa data unit including at least one reference coding unit splitting animage, and the at least one reference coding unit included in theprocessing block may be determined in a certain order. That is, adetermining order of the at least one reference coding unit determinedin each processing block may correspond to one of various orders fordetermining a reference coding unit, and may vary according toprocessing blocks. A determining order of a reference coding unitdetermined per processing block may be one of various orders, such as araster scan order, a Z-scan order, an N-scan order, an up-right diagonalscan order, a horizontal scan order, and a vertical scan order, butshould not be limitedly interpreted by the scan orders.

According to an embodiment, the image decoding device 150 may determinea size of at least one processing block included in an image byobtaining information about a size of a processing block. The imagedecoding device 150 may obtain, from a bitstream, the information abouta size of a processing block to determine the size of the at least oneprocessing block included in the image. The size of the processing blockmay be a predetermined size of a data unit indicated by the informationabout a size of a processing block.

According to an embodiment, the receiver 160 of the image decodingdevice 150 may obtain, from the bitstream, the information about a sizeof a processing block according to certain data units. For example, theinformation about a size of a processing block may be obtained from thebitstream in data units of images, sequences, pictures, slices, andslice segments. That is, the receiver 160 may obtain, from thebitstream, the information about a size of a processing block accordingto such several data units, and the image decoding device 150 maydetermine the size of at least one processing block splitting thepicture by using the obtained information about a size of a processingblock, wherein the size of the processing block may be an integer timesa size of a reference coding unit.

According to an embodiment, the image decoding device 150 may determinesizes of processing blocks 1502 and 1512 included in the picture 1500.For example, the image decoding device 150 may determine a size of aprocessing block based on information about a size of a processingblock, the information obtained from a bitstream. Referring to FIG. 15 ,the image decoding device 150 may determine horizontal sizes of theprocessing blocks 1502 and 1512 to be four times a horizontal size of areference coding unit, and a vertical size thereof to be four times avertical size of the reference coding unit, according to an embodiment.The image decoding device 150 may determine a determining order of atleast one reference coding unit in at least one processing block.

According to an embodiment, the image decoding device 150 may determineeach of the processing blocks 1502 and 1512 included in the picture 1500based on a size of a processing block, and may determine a determiningorder of at least one reference coding unit included in each of theprocessing blocks 1502 and 1512. According to an embodiment, determiningof a reference coding unit may include determining of a size of thereference coding unit.

According to an embodiment, the image decoding device 150 may obtain,from a bitstream, information about a determining order of at least onereference coding unit included in at least one processing block, and maydetermine the determining order of the at least one reference codingunit based on the obtained information. The information about adetermining order may be defined as an order or direction of determiningreference coding units in a processing block. That is, an order ofdetermining reference coding units may be independently determined perprocessing block.

According to an embodiment, the image decoding device 150 may obtain,from a bitstream, information about a determining order of a referencecoding unit according to certain data units. For example, the receiver160 may obtain, from the bitstream, the information about a determiningorder of a reference coding unit according to data units, such asimages, sequences, pictures, slices, slice segments, and processingblocks. Because the information about a determining order of a referencecoding unit indicates a determining order of a reference coding unit ina processing block, the information about a determining order may beobtained per certain data unit including an integer number of processingblocks.

According to an embodiment, the image decoding device 150 may determineat least one reference coding unit based on the determined order.

According to an embodiment, the receiver 160 may obtain, from thebitstream, information about a determining order of a reference codingunit, as information related to the processing blocks 1502 and 1512, andthe image decoding device 150 may determine an order of determining atleast one reference coding unit included in the processing blocks 1502and 1512 and determine at least one reference coding unit included inthe picture 1500 according to a determining order of a coding unit.Referring to FIG. 15 , the image decoding device 150 may determinedetermining orders 1504 and 1514 of at least one reference coding unitrespectively related to the processing blocks 1502 and 1512. Forexample, when information about a determining order of a referencecoding unit is obtained per processing block, determining orders of areference coding unit related to the processing blocks 1502 and 1512 maybe different from each other. When the determining order 1504 related tothe processing block 1502 is a raster scan order, reference coding unitsincluded in the processing block 1502 may be determined according to theraster scan order. On the other hand, when the determining order 1514related to the processing block 1512 is an inverse order of a changedraster scan order, reference coding units included in the processingblock 1512 may be determined in the inverse order of the changed rasterscan order. With reference to FIGS. 1 to 15 , the method of splitting animage into largest coding units, and splitting each largest coding unitinto coding units having a hierarchical tree structure are describedabove. With reference to FIGS. 16 to 25 , it will now be described howto encode or decode the coding units to be split from a current codingunit having a smaller depth than the coding units by 1 according towhich coding order. FIG. 16 illustrates a video decoding device 1600involving determining whether to split a current block and an encodingorder of split lower blocks, according to an embodiment.

The video decoding device 1600 includes a block splitter 1610, anencoding order determiner 1620, a prediction method determiner 1630, anda block decoder 1640. In FIG. 16 , the block splitter 1610, the encodingorder determiner 1620, the prediction method determiner 1630, and theblock decoder 1640 are formed as separate elements, but in anotherembodiment, the block splitter 1610, the encoding order determiner 1620,the prediction method determiner 1630, and the block decoder 1640 may beintegrated to be implemented as one element.

In FIG. 16 , the block splitter 1610, the encoding order determiner1620, the prediction method determiner 1630, and the block decoder 1640are seen as elements located within one apparatus, but the blocksplitter 1610, the encoding order determiner 1620, the prediction methoddeterminer 1630, and the block decoder 1640 are not required to bephysically adjacent to each other. Thus, in another embodiment, theblock splitter 1610, the encoding order determiner 1620, the predictionmethod determiner 1630, and the block decoder 1640 may be dispersed.

The block splitter 1610, the encoding order determiner 1620, theprediction method determiner 1630, and the block decoder 1640 may beimplemented by one processor. In another embodiment, the block splitter1610, the encoding order determiner 1620, the prediction methoddeterminer 1630, and the block decoder 1640 may be implemented by aplurality of processors.

Functions performed by the block splitter 1610, the encoding orderdeterminer 1620, the prediction method determiner 1630, and the blockdecoder 1640 of FIG. 16 may be performed by the decoder 180 of FIG. 1B.

The block splitter 1610 may obtain split information indicating whethera current block is to be split. The split information indicates whetherthe current block is to be split into at least two smaller blocks. Whenthe split information indicates that the current block is to be split,the block splitter 1610 splits the current block into at least two lowerblocks.

The current block may be split into various forms according to a shapeof the current block. For example, when the current block has a squareshape, the current block may be split into at least four square lowerblocks, according to the split information.

When at least two split methods are allowed for the shape of the currentblock, the block splitter 1610 may select a split method according tosplit shape information. Thus, when the split information indicates thatthe current block is to be split, the block splitter 1610 may split thecurrent block, according to the split method indicated by the splitshape information.

For example, when the current block has a square shape of 2N×2N size,the split shape information may indicate a split method from among N×Nsplit, 2N×N split, N×2N split, vertically unequal tri-split, andhorizontally unequal tri-split, the split method being applied to thecurrent block. The N×N split indicates a method of splitting the currentblock into four blocks of N×N size. The 2N×N split indicates a method ofsplitting the current block into blocks of 2N×N size. The N×2N splitindicates a method of splitting the current block into blocks of N×2Nsize. The vertically unequal tri-split indicates a method of splitting a2N×2N-size block into three blocks that have a same width and haverespective heights having a ratio 1:2:1. The horizontally unequaltri-split indicates a method of splitting a 2N×2N-size block into threeblocks that have a same height and have respective heights having aratio 1:2:1. In addition, the current block may be split according toone of various horizontal split methods or vertical split methods.

When the current block has a vertically-long rectangular shape having2N×N size, the split shape information may indicate a split method fromamong N×N split and vertically unequal tri-split, the split method beingapplied to the current block. The N×N split indicates a method ofsplitting the current block into two blocks of N×N size. The verticallyunequal tri-split indicates a method of splitting a 2N×N-size block intothree blocks that have a same width and have respective heights having aratio 1:2:1. In addition, the current block may be split according toone of various horizontal split methods or vertical split methods.

When the current block has a horizontally-long rectangular shape havingN×2N size, the split shape information may indicate a split method fromamong N×N split and horizontally unequal tri-split, the split methodbeing applied to the current block. The N×N split indicates a method ofsplitting the current block into two blocks of N×N size. Thehorizontally unequal tri-split indicates a method of splitting aN×2N-size block into three blocks that have a same height and haverespective heights having a ratio 1:2:1. In addition, the current blockmay be split according to one of various horizontal split methods orvertical split methods.

In addition to the aforementioned split methods, a method ofasymmetrically splitting a current block, a method of splitting acurrent block according to a triangular shape, a method of splitting acurrent block according to other geometric shape, or the like may beused to split a current block having a square shape or a rectangularshape.

When the split information does not indicate that the current block isto be split, the block splitter 1610 does not split the current block.Then, the block decoder 1640 decodes the current block.

When the current block is a coding unit, the block splitter 1610determines the current block as a final coding unit. The final codingunit is not split into coding units having a deeper depth. According toan embodiment, when the current block that is the final coding unit issplit into data units other than a coding unit, the block decoder 1640may make the block splitter 1610 split the current block.

According to an embodiment, the block splitter 1610 may split thecurrent block into one or more prediction units according to ahierarchical tree structure. Equally, the block splitter 1610 may splitthe current block may split the current block into one or more transformunits according to the hierarchical tree structure. Then, the blockdecoder 1640 may reconstruct the current block according to a predictionresult with respect to the prediction units and a transformation resultwith respect to the transform units.

When the current block is a prediction unit, the block decoder 1640 mayperform prediction on the current block. When the current block is atransform unit, the block decoder 1640 may inverse quantize and inversetransform a quantized transform coefficient with respect to the currentblock, thereby obtaining residual data.

The encoding order determiner 1620 obtains encoding order informationindicating an encoding order of lower blocks. Then, the encoding orderdeterminer 1620 may determine a decoding order of the lower blocks,based on the obtained encoding order information.

The encoding order information indicates an encoding order of at leasttwo lower blocks included in the current block. A data amount of theencoding order information is determined based on the number of lowerblocks and an encoding order determining scheme.

For example, when there are two lower blocks, the encoding orderinformation may be determined to indicate a first-encoded lower blockfrom among the two lower blocks. Thus, the encoding order informationmay be in the form of a flag having a 1-bit data amount. However, whenthere are four lower blocks, the number of cases of an encoding order oflower blocks is 4!=24. Therefore, to indicate 24 encoding orders, a5-bit data amount is required. That is, when the number of lower blocksis increased, the number of cases of an encoding order is increased.Therefore, to decrease a data amount of the encoding order information,an encoding order determining scheme of determining an encoding order bydetermining whether encoding orders of some lower block pairs areswapped in a predetermined default encoding order. Encoding orderinformation indicating whether the encoding orders of some lower blockpairs are swapped indicates a forward direction or a backward directionwith respect to the default encoding order.

A current picture including the current block is encoded and decodedaccording to the default encoding order. All blocks and pixels to beencoded and decoded in the current picture are to be encoded and decodedat a same level according to the default encoding order. Thus, lowerblocks at a same level split from the current block are also to beencoded and decoded according to the default encoding order. Anembodiment of the default encoding order is illustrated in FIGS. 17A to17C to be described below.

Therefore, when a lower block pair is encoded according to the defaultencoding order, it is described that the lower block pair is encoded ina forward direction. On the contrary, when the lower block pair isencoded according to an inverse order to the default encoding order, itis described that the lower block pair is encoded in a backwarddirection.

For example, in a case where two lower blocks are horizontally adjacentto each other and are encoded in a forward direction, the encoding orderinformation may be determined to allow a left lower block to be firstdecoded. On the contrary, in a case where the two lower blocks that arehorizontally adjacent to each other are encoded in a backward direction,the encoding order information may be determined to allow a right lowerblock to be first decoded.

Equally, in a case where two lower blocks are vertically adjacent toeach other and are encoded in a forward direction, the encoding orderinformation may be determined to allow an upper lower block to be firstdecoded. On the contrary, in a case where the two lower blocks that arevertically adjacent to each other are encoded in a backward direction,the encoding order information may be determined to allow a furtherlower block to be first decoded.

When the encoding order information indicates only an encoding order ofa lower block pair, the encoding order information has a 1-bit dataamount. The encoding order information having 1-bit data amount may bedefined as an encoding order flag.

The encoding order determiner 1620 may obtain the encoding orderinformation from a bitstream. The encoding order information may bepositioned after split information in the bitstream.

The encoding order determiner 1620 may implicitly determine the encodingorder information according to a surrounding environment of the currentblock. The encoding order information may be determined according towhether neighboring blocks adjacent to the current block have beenencoded. For example, the encoding order determiner 1620 may determine alower block to be first decoded, the lower block having many adjacentneighboring blocks from among lower blocks.

With respect to the encoding order determiner 1620, a default encodingorder according to an embodiment will now be described with reference toFIGS. 17A to 17C. The default encoding order of FIGS. 17A to 17C is a Zencoding order. According to the Z encoding order, data units areencoded from the left to the right, and when data units of a current roware all encoded, data units included in a lower row of the current roware encoded from the left to the right. The aforementioned Z encodingorder is referred to as a raster scan order.

FIG. 17A illustrates encoding orders according to a Z encoding order oflargest coding units included in a current picture 1700. According tothe Z encoding order, indexes 0 to 15 are set to the largest codingunits. Largest coding units of a first row to which the indexes 0 to 3are set according to the Z encoding order are first encoded, and largestcoding units of a second row to which the indexes 4 to 7 are encodedfrom the left to the right. The largest coding units are internallyencoded according to the Z encoding order.

FIG. 17B illustrates an encoding order of a largest coding unit 1710having the index 6 from among the largest coding units included in thecurrent picture 1700. Coding units of a final depth for which split hasbeen completed according to the Z encoding order are set with theindexes 0 to 15. The Z encoding order is applied to data units of a samedepth. In addition, until lower coding units of a current coding unit ofa depth n are all encoded, a next coding unit of a depth n is notencoded. For example, until coding units having the indexes 5 to 14 areall encoded, a coding unit having the index 15 is not encoded. Thecoding units are also internally encoded according to the Z encodingorder.

FIG. 17C illustrates a reference sample to be referred to by a codingunit 1724 having the index 6 from among the coding units included in thelargest coding unit 1710. Only a coding unit 1712 having the index 0 anda coding unit 1722 having the index 5 have been reconstructed around thecoding unit 1724 having the index 6 to be currently encoded. Therefore,for the coding unit 1724, only a pixel 1750 of the coding unit 1712 anda pixel 1760 of the coding unit 1722 may be used as a reference sample.

The Z encoding order of FIGS. 17A to 17C may be applied in anotherdirection according to a data unit. For example, the Z encoding ordermay be changed to allow data units to be encoded from the right to theleft in a same row. Also, the Z encoding order may be changed such that,after all data units of a current row are encoded, data units includedin an upper row of the current row are to be encoded. Also, the Zencoding order may be changed such that data units of a same column areencoded from the top to the bottom and, after all data units of acurrent column are encoded, data units included in a right column of thecurrent column are to be encoded.

Regarding the encoding order determiner 1620, FIGS. 18A and 18Brespectively illustrate a case 1800 in which a coding unit 1810 isencoded in a forward direction and a case 1802 in which a coding unit1820 is encoded in a backward direction. With reference to FIGS. 18A and18B, an advantage obtained by changing an encoding order will now bedescribed.

The coding units 1810 and 1820 of FIGS. 18A and 18B are predictedaccording to an intra mode in an upper right direction. A continuousline 1830 of FIGS. 18A and 18B corresponds to pixels having a constantvalue and arranged in a straight line in an original image. Therefore,when a current coding unit is predicted in a direction of the continuousline 1830, prediction accuracy with respect to the coding units 1810 and1820 may be improved.

In the case 1800 of encoding in the forward direction, a left codingunit, an upper coding unit, and an upper right coding unit of thecurrent coding unit 1810 are first reconstructed before the currentcoding unit 1810. Therefore, the current coding unit 1810 refers topixels or encoding information of the left coding unit, the upper codingunit, and the upper right coding unit. For example, pixels 1816 locateda lower corner of the upper right coding unit are used in predicting thecurrent coding unit 1810. Because the pixels 1816 are spatially distantfrom the current coding unit 1810, prediction accuracy with respect to aportion 1814 of the current coding unit 1810 may be low.

However, in the case 1802 of encoding in the inverse direction, a rightcoding unit, an upper coding unit, and an upper left coding unit of acurrent coding unit 1820 are first reconstructed before the currentcoding unit 1820, and thus, in intra prediction, pixels 1826 located ata left corner of the right coding unit may be used in predicting thecurrent coding unit 1820. Because the pixels 1826 are adjacent to thecurrent coding unit 1820, prediction accuracy with respect to a portion1824 of the current coding unit 1820 may be further improved than theprediction accuracy with respect to the portion 1814 of the currentcoding unit 1810.

As in an embodiment of the intra prediction described with reference toFIGS. 18A and 18B, there are many cases in which prediction accuracy ofinter prediction may be improved by obtaining encoding information froma block located in a backward direction. When a current coding unit anda right coding unit of the current coding unit are coding units withrespect to a same object, the current coding unit and motion informationof the right coding unit may be similar to each other. Therefore, codingefficiency may be increased by deriving motion information of thecurrent coding unit from the motion information of the right codingunit.

Therefore, by determining an encoding order by comparing codingefficiency of a case in which the current coding unit is encoded in aforward direction with coding efficiency of a case in which the currentcoding unit is encoded in a backward direction, coding efficiency withrespect to an image may be improved.

Encoding order information may be set to be equal to encoding orderinformation applied to an upper block of a current block. For example,when the current block is a prediction unit or a transform unit, theencoding order determiner 1620 may apply, to the current block, encodingorder information applied to a coding unit including the current block.As another example, when the current block is a coding unit, theencoding order determiner 1620 may apply, to the current block, encodingorder information applied to a coding unit whose depth is lower than thecurrent block.

When at least two encoding order flags are present with respect to thecurrent block, the encoding order determiner 1620 may obtain only oneencoding order flag from a bitstream, and may determine the otherencoding order flag to interoperate with the encoding order flagobtained from the bitstream.

With respect to encoding order determination by the encoding orderdeterminer 1620, FIG. 19 illustrates a tree structure of a largestcoding unit for describing an encoding order of the largest coding unitand coding units included in the largest coding unit.

A largest coding unit 1950 is split into a plurality of coding units1956, 1958, 1960, 1962, 1968, 1970, 1972, 1974, 1980, 1982, 1984, and1986. The largest coding unit 1950 corresponds to an uppermost node 1900of the tree structure. The plurality of coding units 1956, 1958, 1960,1962, 1968, 1970, 1972, 1974, 1980, 1982, 1984, and 1986 respectivelycorrespond to a plurality of nodes 1906, 1908, 1910, 1912, 1918, 1920,1922, 1924, 1930, 1932, 1934, and 1936. Upper encoding order flags 1902,1914, and 1926 indicating encoding orders in the tree structurecorrespond to arrows 1952, 1964, and 1976, and upper encoding orderflags 1904, 1916, and 1928 correspond to arrows 1954, 1966, and 1978.

An upper encoding order flag indicates an encoding order of two codingunits located above from among four coding units having a same depth.When the upper encoding order flag indicates 0, encoding is performed ina forward direction. On the contrary, when the upper encoding order flagindicates 1, encoding is performed in a backward direction.

Equally, a lower encoding order flag indicates an encoding order of twocoding units located in the lower side from among the four coding unitshaving the same depth. When the lower encoding order flag indicates 0,encoding is performed in a forward direction. On the contrary, when thelower encoding order flag indicates 1, encoding is performed in abackward direction.

For example, because an upper encoding order flag 1914 indicates 0, anencoding order between coding units 1968 and 1970 is determined to befrom the left that is a forward direction to the right. Also, because alower encoding order flag 1916 indicates 0, an encoding order betweencoding units 1972 and 1974 is determined to be from the right that is abackward direction to the left.

According to an embodiment, it may be set for an upper encoding orderflag and a lower encoding order flag to have a same value. For example,when the upper encoding order flag 1902 is determined to be 1, the lowerencoding order flag 1904 corresponding to the upper encoding order flag1902 may be determined to be 1. Because values of the upper encodingorder flag and the lower encoding order flag are determined to be 1 bit,information amount of encoding order information is decreased.

According to an embodiment, an upper encoding order flag and a lowerencoding order flag of a current coding unit may be determined byreferring to at least one of an upper encoding order flag and a lowerencoding order flag applied to a coding unit having a depth lower thanthe current coding unit. For example, the upper encoding order flag 1926and the lower encoding order flag 1928 applied to the coding units 1980,1982, 1984, and 1986 may be determined based on the lower encoding orderflag 1916 applied to the coding units 1972 and 1974. Therefore, theupper encoding order flag 1926 and the lower encoding order flag 1928may be determined to have a same value as the lower encoding order flag1916. Because values of the upper encoding order flag and the lowerencoding order flag are determined from an upper coding unit of thecurrent coding unit, encoding order information is not obtained from abitstream. Therefore, information amount of the encoding orderinformation is decreased.

With respect to encoding order determination by the encoding orderdeterminer 1620, how an encoding order of at least three blocks arrangedin a vertical or horizontal direction is changed according to anencoding order flag will now be described with reference to FIGS. 20Aand 20B.

An embodiment of FIG. 20A is about a method of swapping encoding orders,based on an encoding order flag, only when the encoding orders ofspatially-adjacent coding units are adjacent to each other.

A coding unit 2000 is split into three coding units 2010, 2020, and2030. When a default encoding order is from the left to the right,encoding is performed in order of the coding unit 2010, the coding unit2020, and the coding unit 2030. However, an encoding order may bechanged according to encoding order flags 2040 and 2050.

The encoding order flag 2040 indicates an encoding order of the codingunit 2010 and the coding unit 2020. When the encoding order flag 2040indicates 0, the encoding order of the coding unit 2010 and the codingunit 2020 is determined to be a forward direction. Therefore, the codingunit 2010 is encoded prior to the coding unit 2020. However, when theencoding order flag 2040 indicates 1, the encoding order of the codingunit 2010 and the coding unit 2020 is determined to be a backwarddirection, and thus the coding unit 2020 is encoded prior to the codingunit 2010.

The encoding order flag 2050 indicates an encoding order of the codingunit 2020 and the coding unit 2030. When the encoding order flag 2040indicates a forward direction, the encoding order flag 2050 is obtained.When the encoding order flag 2040 indicates a backward direction,encoding orders of the coding unit 2020 and the coding unit 2030 are notadjacent to each other, and thus the encoding order flag 2050 is notobtained. When the encoding order flag 2050 indicates 0, an encodingorder of the coding unit 2020 and the coding unit 2030 is determined tobe a forward direction. Therefore, the coding unit 2020 is encoded priorto the coding unit 2030. However, when the encoding order flag 2050indicates 1, an encoding order of the coding unit 2020 and the codingunit 2030 is determined to be a backward direction, and thus the codingunit 2030 is encoded prior to the coding unit 2020.

According to an embodiment of FIG. 20A, an encoding order of threecoding units has three cases. Therefore, to determine the encodingorder, one or two encoding order flags are used.

An embodiment of FIG. 20B is about a method of determining an encodingorder, based on an encoding order flag 2060 indicating a direction ofthe encoding order to be applied to three coding units.

The encoding order flag 2060 indicates whether an encoding order is aforward direction or a backward direction. For example, when theencoding order flag 2060 indicates 0, an encoding order of the codingunits 2010, 2020, and 2030 may be determined to be the forwarddirection. Therefore, when the encoding order flag 2060 indicates 0,encoding may be performed in order of the coding unit 2010, the codingunit 2020, and the coding unit 2030.

On the other hand, when the encoding order flag 2060 indicates 1, theencoding order of the coding units 2010, 2020, and 2030 may bedetermined to be the backward direction. Therefore, when the encodingorder flag 2060 indicates 1, encoding may be performed in order of thecoding unit 2030, the coding unit 2020, and the coding unit 2010.

Referring to the embodiment of FIG. 20B, the encoding order of threecoding units has two cases. Therefore, to determine the encoding order,one encoding order flag is used.

The methods of determining an encoding order which are used in theembodiments of FIGS. 20A and 20B may be applied to at least four codingunits.

The encoding order determiner 1620 may check encoding order changeallowance information with respect to an upper data unit of a currentblock. The encoding order change allowance information indicates whethera change in an encoding order is allowable for blocks included in theupper data unit of the current block. When the encoding order changeallowance information indicates that the change in the encoding order isnot allowable, all blocks of the upper data unit are decoded accordingto a default encoding order. Alternatively, all blocks of the upper dataunit may be decoded according to an encoding order of a data unitincluding the upper data unit. When the encoding order change allowanceinformation indicates that encoding order information with respect tothe current block has been encoded, the encoding order determiner 1620may obtain the encoding order information.

The encoding order change allowance information may be included in avideo parameter set, a sequence parameter set, a picture parameter set,a slice segment header, a header of a largest coding unit, or the like.When at least two types of the encoding order information are present,two pieces of encoding order change allowance information about at leasttwo types of the encoding order information may be separately stored indifferent headers.

The encoding order change allowance information may indicate a depth atwhich encoding order information is provided, or a block size. Forexample, only when a depth of the current block is included in the depthindicated by the encoding order change allowance information, theencoding order determiner 1620 may obtain the encoding orderinformation. As another example, only when the depth of the currentblock corresponds to the block size indicated by the encoding orderchange allowance information, the encoding order determiner 1620 mayobtain the encoding order information.

When split information does not indicate that the current block is to besplit, the prediction method determiner 1630 may determine a predictionmethod with respect to the current block according to encodinginformation of the current block and whether neighboring blocks of thecurrent block have been decoded.

The encoding information of the current block may indicate how thecurrent block is to be predicted. In detail, the encoding informationmay indicate a prediction method from among a plurality of intraprediction modes and inter prediction modes. The intra prediction modesthat are applicable to the current block may include a p directionalmode, a DC mode, a planar mode, a multi-parameter intra (MPI) predictionmode, a linear-model (LM) chroma mode, a most probable chroma (MPC)mode, or the like. The inter prediction modes that are applicable to thecurrent block may include a merge mode, an advanced motion vectorprediction (AMVP) mode, an inter skip mode, an overlapped block motioncompensation (OBMC) mode, a sub-block motion vector prediction (MVP)mode, an affine motion compensation (MC) mode, a frame rate upconversion (FRUC) mode, or the like. Therefore, the prediction methoddeterminer 1630 may determine a prediction mode to be applied to thecurrent block, based on the encoding information of the current block.

Whether the neighboring blocks of the current block have been decoded, areference block and a reference sample to be referred to in predictingthe current block may be determined. Referring to the raster scandescribed with reference to FIGS. 17A to 17C, only left, upper, upperleft, upper right, and lower left blocks in the current block may havebeen decoded prior to the current block. However, when an encoding treeblock including the current block has been decoded by the encoding orderdeterminer 1620 according to an encoding order different from the rasterscan, a right block and a lower right block of the current block mayhave been decoded prior to the current block. Therefore, the predictionmethod determiner 1630 may determine the reference block and thereference sample to be referred to in predicting the current block,according to whether the left, upper, upper left, upper right, lowerleft, right, and lower right blocks of the current block have beendecoded.

When the current block is intra predicted, the prediction methoddeterminer 1630 may determine reference samples to be referred to forthe current block, according to whether the neighboring blocks of thecurrent block have been decoded. In an intra prediction mode, predictionvalues of samples of the current block are determined by referring tosample values of samples adjacent to the current block. Therefore, onlyan adjacent block from among adjacent blocks of the current block may beused in predicting the current block, wherein the adjacent block hasbeen decoded prior to the current block and may be referred to for thecurrent block. For example, when blocks are encoded according to aforward direction of the raster scan described with reference to FIGS.17A to 17C, reference samples of the upper block, the left block, theupper left block, the lower left block, and the upper right block of thecurrent block may be used in predicting the current sample. On thecontrary, when the blocks are encoded according to a backward directionto the raster scan, reference samples of the upper block, the rightblock, the upper right block, the lower right block, and the upper leftblock of the current block may be used in predicting the current sample.

A method of using reference samples according to a direction of anencoding order, the method being related to the prediction methoddeterminer 1630, will now be described in detail with reference to FIGS.21 to 25B.

FIG. 21 illustrates a method of determining a reference sample requiredin a directional intra prediction mode.

A first embodiment 2120 illustrates reference samples 2102, 2106, 2108,and 2110 used in intra prediction when blocks in an upper row and a leftblock are reconstructed. In the first embodiment 2120, the referencesamples 2102 and 2106 of the reconstructed upper blocks and thereference samples 2108 of the reconstructed left block may be used inthe intra prediction. The reference samples 2110 of a lower left blockmay be used only when the lower left block is reconstructed. To use thereference samples 2102, 2106, 2108, and 2110, prediction directionsincluded in a first intra prediction direction group 2125 may be used inintra predicting a current block 2100.

A second embodiment 2130 illustrates reference samples 2102, 2104, 2112,and 2114 used in intra prediction when blocks in an upper row and aright block are reconstructed. In the second embodiment 2130, thereference samples 2102 and 2104 of the reconstructed upper blocks andthe reference samples 2112 of the reconstructed right block may be usedin the intra prediction. The reference samples 2114 of a lower rightblock may be used only when the lower right block is reconstructed. Touse the reference samples 2102, 2104, 2112, and 2114, predictiondirections included in a second intra prediction direction group 2135may be used in intra predicting the current block 2100.

A third embodiment 2140 illustrates reference samples 2102, 2108, and2112 used in intra prediction when an upper block, a right block, and aleft block are reconstructed. In the third embodiment 2140, thereference samples 2102 of the upper block, the reference samples 2108 ofthe left block, and the reference samples 2112 of the right block may beused in the intra prediction. Prediction directions included in a thirdintra prediction direction group 2145 may be used in intra predictingthe current block 2100.

According to the first embodiment 2120 and the second embodiment 2130,when the reference samples 2110 of the lower left block and thereference samples 2114 of the lower right block cannot be used,prediction accuracy may deteriorate. However, in the third embodiment2140, the used reference samples 2102, 2108, and 2112 are all adjacentto the current block 2100, and thus, prediction accuracy may berelatively high, compared to other embodiments.

A fourth embodiment 2150 illustrates reference samples 2102, 2104, and2106 used in intra prediction when only blocks in an upper row arereconstructed. In the fourth embodiment 2150, only the reference samples2102, 2104, and 2106 of the reconstructed upper blocks may be used inthe intra prediction. Prediction directions included in a fourth intraprediction direction group 2155 may be used in intra predicting thecurrent block 2100.

Unlike the third embodiment 2140, in the fourth embodiment 2150, thereference sample 2102 of the upper block is the only sample that isadjacent to the current block 2100. Because the reference samples 2104and 2106 are spatially distant from the current block 2100, predictionaccuracy may deteriorate, compared to the first, second, and thirdembodiments 2120, 2130, and 2140. Therefore, the intra prediction usedin the fourth embodiment 2150 may be a vertical mode or a directionalprediction mode in a direction adjacent to the vertical mode which usesthe reference sample 2102 of the upper block that is adjacent to thecurrent block 2100.

In a Z encoding order, the inter prediction method according to thefirst embodiment 2120 is used, but, when encoding orders of two blocksthat are adjacent in a left-right direction have been swapped, the rightblock may be first predicted according to the intra prediction methodaccording to the fourth embodiment 2150. Then, after the right block isreconstructed, the left block may be predicted according to the intraprediction method according to the third embodiment 2140 and thus may bereconstructed.

FIGS. 22A and 22B illustrate a prediction method in a DC mode accordingto whether a right block has been decoded. FIG. 22A indicates a range ofreference samples when an encoding order of a current block is fixed toa default encoding order according to a raster scan. FIG. 22B indicatesa range of reference samples when an encoding order of the current blockis not fixed to the default encoding order.

Referring to FIG. 22A, decoded samples from among samples 2210 that areadjacent to left, upper, and upper left sides of a current block 2200may be used as a reference sample. However, in FIG. 22B, a right blockof a current block 2230 may have been decoded prior to the current block2230, and thus decoded samples from among samples 2240 that are adjacentto left, upper, upper left, upper right, and right sides of the currentblock 2230 may be used as a reference sample.

In a DC mode, all samples of a current block have a same predictionsample, and thus a discontinuity may occur between a sample value of areference sample and a prediction value of a sample of the currentblock. Therefore, in the DC mode, a prediction value of a sample locatedat a boundary of the current block is filtered according to an adjacentreference sample, and thus continuity in the prediction value of thesample of the current block may be achieved. Therefore, predictionaccuracy in the DC mode may be increased.

Referring to FIG. 22A, only left, upper, and upper left samples of thecurrent block 2200 may be determined as the reference sample, and thusprediction values of samples 2220 located at left and top boundaries ofthe current block 2200 are targets for prediction value filtering.However, referring to FIG. 22B, right and upper right samples of thecurrent block 2230 may be determined as the reference sample, and thusprediction values of samples 2250 located at left, top, and rightboundaries of the current block 2230 may be targets for prediction valuefiltering.

FIGS. 23A to 23C illustrate a prediction method in a planar modeaccording to whether a right block has been decoded. FIG. 23A indicatesa method of determining a prediction value of a current sample in theplanar mode when an encoding order of a current block is fixed to adefault encoding order according to a raster scan. FIGS. 23B and 23Cindicate a method of determining a prediction value of the currentsample in the planar mode when an encoding order of the current block isnot fixed to the default encoding order.

Referring to FIG. 23A, double interpolation values of four referencesamples 2302, 2304, 2306, and 2308 with respect to a current sample 2300are determined as a prediction value of the current sample 2300. Ahorizontal interpolation value is determined by performing linearinterpolation on a first corner sample 2302 and a first side sample2304, based on a horizontal location of the current sample 2300. Avertical interpolation value is determined by performing linearinterpolation on a second corner sample 2306 and a second side sample2308, based on a vertical location of the current sample 2300. Anaverage value of the horizontal interpolation value and the verticalinterpolation value is determined as the prediction value of the currentsample 2300.

With reference to FIG. 23B, a prediction method with respect to acurrent sample in a planar mode, when a left block of a current blockhas not been decoded and a right block of the current block has beendecoded, will now be described.

Referring to FIG. 23B, four reference samples 2312, 2314, 2316, and 2318that are symmetrical to FIG. 23A are determined. A prediction method ina planar mode of FIG. 23B is different from that of FIG. 23A inlocations of reference samples, but a method of determining a predictionvalue of a current sample is similar to that of FIG. 23A.

In detail, a horizontal interpolation value is determined by performinglinear interpolation on a first corner sample 2312 and a first sidesample 2314, based on a horizontal location of a current sample 2310. Avertical interpolation value is determined by performing linearinterpolation on a second corner sample 2316 and a second side sample2318, based on a vertical location of the current sample 2310. Then, anaverage value of the horizontal interpolation value and the verticalinterpolation value is determined as the prediction value of the currentsample 2310.

With reference to FIG. 23C, a prediction method with respect to acurrent sample in a planar mode, when a left block and a right block ofthe current block have been all decoded, will now be described.

Referring to FIG. 23C, because reference samples are present in the top,left, and right sides of the current block, prediction accuracy in theplanar mode may be increased. However, because the bottom outside thecurrent block has not been decoded, a first corner sample 2328 and asecond corner sample 2330 are linear interpolated according to ahorizontal location of a current sample 2320 and thus a bottom mediumvalue for calculation of a vertical interpolation value is determined.

A horizontal interpolation value is determined by interpolating a firstside sample 2322 and a second side sample 2324, based on a horizontaldistance of the current sample 2320. Also, a vertical interpolationvalue is determined by interpolating a third side sample 2326 and thebottom medium value, based on a vertical location of the current sample2320. Then, an average value of the horizontal interpolation value andthe vertical interpolation value is determined as the prediction valueof the current sample 2310.

Also, as seen in FIG. 23C, when both a left block and a right block of acurrent block have been decoded, one of the prediction method in aplanar mode of FIG. 23A and the prediction method in a planar mode ofFIG. 23B may be selected, and the current block may be predictedaccording to the selected prediction method in a planar mode.Alternatively, when both the left block and the right block of thecurrent block have been decoded, a prediction value according to theplanar mode may be determined by averaging a prediction value obtainedaccording to a method of FIG. 24A and a prediction value obtainedaccording to a method of FIG. 24B.

Alternatively, when both the left block and the right block of thecurrent block have been decoded, a current sample located in aparticular line of the current block may be predicted by interpolatingthe side samples 2304 and 2314 located in the left and right, accordingto a location of a current sample.

FIGS. 24A to 24D illustrate a method of predicting a current blockaccording to a multi-parameter intra (MPI) mode. The MPI mode is amethod of predicting a current sample by using samples that are fromamong neighboring samples of the current sample and are decoded orpredicted in a particular direction. In particular, the current sampleis determined as a weighted average value of sample values ofneighboring samples, and a weight used in the weighted average value maybe predicted from a neighboring block or may be obtained from abitstream. Sample values of neighboring blocks which are used inpredicting the current block may be prediction values or reconstructionvalues of the neighboring blocks.

FIG. 24A illustrates a method of predicting a current block according tothe MPI mode when an encoding order of the current block is fixed to adefault encoding order according to a raster scan. FIGS. 24B to 24Dillustrate a method of predicting a current block according to the MPImode when an encoding order of the current block is not fixed to thedefault encoding order.

Referring to FIG. 24A, a prediction value of a current sample includedin a current block 2400 is determined as an average value or a weightedaverage value which is obtained by averaging or weight-averaging asample value of a left sample of the current sample and a sample valueof an upper sample of the current sample. Referring to FIG. 24A, becausereference samples are located in the left and top of the current block2400, an upper left sample of the current block 2400 is first predicted.

For example, a prediction value of a sample 2402 of the current block2400 is determined as a weighted average value of sample values of areference sample 2412 and a reference sample 2414. The prediction valueor a reconstruction value of the sample 2402 is used in predicting asample 2404 in the right of the sample 2402 and a sample 2406 below thesample 2402. Therefore, a prediction value of the sample 2404 isdetermined as a weighted average value of sample values of the sample2402 and a reference sample 2416, and a prediction value of the sample2406 is determined as a weighted average value of sample values of thesample 2402 and a reference sample 2418. In this regard, a weight usedin determining the weighted average value may be determined according toa location of the current sample. Also, other samples of the currentblock 2400 are predicted based on the weight.

Referring to FIG. 24B, opposite to FIG. 24A, an upper block and a rightblock of a current block 2420 have been first decoded, and a left blockof the current block 2420 is not decoded. Therefore, referring to FIG.24B, a prediction value of a current sample included in the currentblock 2420 is determined as an average value or a weighted average valuewhich is obtained by averaging or weight-averaging a sample value of aright sample in the current sample and a sample value in an upper sampleof the current sample. Referring to FIG. 24B, because reference samplesare located in the right and top of the current block 2420, an upperright sample in the current block 2420 is first predicted.

For example, a prediction value of a sample 2422 of the current block2420 is determined as a weighted average value of sample values of areference sample 2432 and a reference sample 2434. The prediction valueor a reconstruction value of the sample 2422 is used in predicting asample 2424 in the left of the sample 2422 and a sample 2426 below thesample 2422. Therefore, a prediction value of the sample 2424 isdetermined as a weighted average value of sample values of the sample2422 and a reference sample 2436, and a prediction value of the sample2426 is determined as a weighted average value of sample values of thesample 2422 and a reference sample 2438. In this regard, a weight usedin determining the weighted average value may be determined according toa location of the current sample. Also, other samples of the currentblock 2420 are predicted in a same manner.

Referring to FIG. 24C, only an upper block of a current block 2440 hasbeen first decoded, and a right block and a left block of the currentblock 2440 have not been decoded. Therefore, referring to FIG. 24C, onlyreference samples in the top of a current block are used to predictsamples of the current block. In a case of FIG. 24C, equally to FIG.24A, the current block may be predicted in a lower right direction froman upper left sample of the current block. Also, equally to FIG. 24B,the current block may be predicted in a lower right direction from anupper right sample of the current block. Therefore, when only an upperblock of the current block has been first decoded, a prediction orderand a prediction method with respect to samples may be determined basedon neighboring blocks of the current block or may be obtained from abitstream.

FIG. 24C illustrates a case in which the current block 2440 is predictedin a lower right direction from an upper left sample in the currentblock 2440. Referring to FIG. 24C, a method of predicting the currentblock 2440 is similar to that of FIG. 24A. However, unlike to FIG. 24A,samples adjacent to a left boundary of the current block 2440 do nothave reference samples in the left, and thus, the samples are predictedbased on only a sample value of an upper sample.

For example, a prediction value of a sample 2442 of the current block2440 is determined as a sample value of a reference sample 2452. Theprediction value or a reconstruction value of the sample 2442 is used inpredicting a sample 2444 in the right of the sample 2442 and a sample2446 below the sample 2442. A prediction value of the sample 2446located at a left boundary of the current block 2440 is determined basedon the prediction value or the reconstruction value of the sample 2442,and samples below the sample 2446 are predicted in a same manner.Samples located in other portions of the current block 2440 arepredicted in a same manner as in FIG. 24A.

Referring to FIG. 24D, an upper block, a right block, and a left blockof a current block 2460 have been all decoded. Therefore, referring toFIG. 24D, reference samples in the top, right, and left of the currentblock 2460 may be all used to predict samples of the current block 2460.Accordingly, referring to FIG. 24D, the current block 2460 may bepredicted by using various schemes according to the reference samples inthe top, right, and left of the current block 2460.

According to an embodiment provided in FIG. 24D, the current block 2460is divided into a left sub-block 2470 and a right sub-block 2480. Theleft sub-block 2470 is predicted in a lower right direction from anupper left sample, and the right sub-block 2480 is predicted in a lowerleft direction from an upper right sample as in FIG. 24B. In detail, asample 2472 located at a top left corner of the left sub-block 2470 isdetermined as an average value or a weighted average value of a topreference sample 2490 and a left reference sample 2492. Other samples ofthe left sub-block 2470 are each determined as an average value or aweighted average value of sample values of left and upper samples. Asample 2482 located at a top right corner of the right sub-block 2480 isdetermined as an average value or a weighted average value of a topreference sample 2494 and a right reference sample 2496. Other samplesof the right sub-block 2480 are each determined as an average value or aweighted average value of sample values of right and upper samples.

With reference to FIGS. 25A and 25B, reference areas that are referredto in an LM chroma mode and an MPC mode will now be described. The LMchroma mode and the MPC mode are prediction modes for predicting achroma block from a collocated luma block. In the LM chroma mode and theMPC mode, a reference area in which a luma sample and a chroma samplehave been all decoded is determined, a correlation between the lumasample and the chroma sample is obtained from the reference area, andthe chroma block is predicted based on a sample value of the luma blockcorresponding to the chroma block and the correlation between the lumasample and the chroma sample.

However, in the LM chroma mode, a luma-chroma linear model is derivedfrom a correlation between a sample value of the luma sample and asample value of the chroma sample which are obtained from the referencearea, and then the chroma block is predicted from the collocated lumablock, according to the luma-chroma linear model. In the MPC mode, amost probable chroma (MPC) value with respect to each luma sample valueis determined from the correlation between the sample value of the lumasample and the sample value of the chroma sample which are obtained fromthe reference area, and then the chroma block is predicted from thecollocated luma block, according to a result of analyzing the MPC value.

FIG. 25A illustrates a reference area of the LM chroma mode and the MPCmode when an encoding order of a current block is fixed to a defaultencoding order according to a raster scan. FIG. 25B illustrates areference area of the LM chroma mode and the MPC mode when an encodingorder of the current block is not fixed to the default encoding order.

Referring to FIG. 25A, left, upper, and upper left blocks of a currentblock 2500 have been decoded. Therefore, a reference area 2510 is set inthe current block 2500 in left, upper, and upper left directions.

Referring to FIG. 25B, opposite to FIG. 25A, right, upper, and upperright blocks of a current block 2520 have been decoded. Therefore, areference area 2530 is set in the current block 2520 in right, upper,and upper right directions.

Unlike to FIGS. 25A and 25B, when all of left, upper, right, upper left,and upper right blocks of a current block have been decoded, a referencearea may be set in the current block in left, upper, right, upper left,and upper right directions.

Widths of the reference areas 2510 and 2530 of FIGS. 25A and 25B aredetermined according to a reference area offset. The reference areaoffset may be determined based on sizes of the current blocks 2500 and2520, or may be determined based on encoding information obtained from abitstream.

Other than the intra prediction mode described with reference to FIGS.21 to 25B, when a right block has been decoded, the prediction methoddeterminer 1630 may predict a current block by using a right referencesample instead of a left block or with the left block.

When the current block is inter predicted, the prediction methoddeterminer 1630 may determine a reference block to be referred to forthe current block, according to whether a neighboring block of thecurrent block has been decoded. In the inter prediction mode, theprediction method determiner 1630 obtains a motion vector from a blockthat is spatially or temporally adjacent to the current block, andpredicts the current block according to the obtained motion vector.

A reference picture including a temporally-adjacent block is decodedprior to a current picture. Therefore, regardless of an encoding orderof blocks, temporally-adjacent blocks may be determined in a samemanner. However, some of spatially-adjacent blocks may have not beendecoded according to the encoding order of blocks. Therefore, aspatially-adjacent block to be referred to for the current block mayvary according to a decoding order.

In detail, when the blocks are encoded according to a forward directionof the raster scan described with reference to FIGS. 17A to 17C, motionvectors of an upper block, a left block, an upper left block, a lowerleft block, and an upper right block of the current block may be used inpredicting a current sample. On the contrary, when the blocks areencoded in an opposite direction to the raster scan, reference samplesof the upper block, a right block, an upper right block, a lower rightblock, and the upper left block of the current block may be used inpredicting the current sample.

When the left and right blocks of the current block have been alldecoded, the upper block, the left block, the right block, the upperleft block, the lower left block, the lower right block, and the upperright block may be all used. On the contrary, when the left and rightblocks of the current block have not been decoded, only the upper block,the upper left block, and the upper right block may be used.

With reference to FIG. 26 , a block that is spatially adjacent to acurrent block according to an encoding order of the current block in amerge mode and an AMVP mode will now be described.

The merge mode is an inter prediction mode in which a reference block ofa current block is determined from a merge candidate list consisting ofneighboring blocks of the current block, and the current block ispredicted based on a motion vector extracted from the reference blockand a reference picture index. The merge candidate list may bedetermined according to different schemes according to whether themotion vector is extractable from a right block.

When the motion vector is not extractable from the right block, theprediction method determiner 1630 may extract motion vectors, in orderof a left block including a reference sample 2604, a lower left blockincluding a reference sample 2606, an upper block including a referencesample 2608, an upper right block including a reference sample 2610, andan upper left block including a reference sample 2602. Then, theprediction method determiner 1630 may sequentially include the extractedmotion vectors in the merge candidate list.

When the motion vector is extractable from the right block, theprediction method determiner 1630 may extract motion vectors, in orderof a right block including a reference sample 2614, a lower right blockincluding a reference sample 2616, an upper block including a referencesample 2612, an upper left block including a reference sample 2602, aleft block including a reference sample 2604, and an upper right blockincluding a reference sample 2610. Then, the prediction methoddeterminer 1630 may sequentially include the extracted motion vectors inthe merge candidate list.

When at most 5 merge candidates can be included in the merge candidatelist, only inter-predicted blocks from among the 6 blocks may besequentially included in the merge candidate list. Therefore, in a casewhere only the right block is decoded and inter-predicted whereas theleft block is not, only the left block is excluded from the mergecandidate list, and the right block, the lower right block, the upperblock, the upper left block, and the upper right block may be includedtherein.

In the AMVP mode, a motion vector predictor is obtained from thereference block, and a difference motion vector and a reference pictureindex are separately obtained from a bitstream. Then, the current blockis predicted based on the motion vector and the reference picture indexthat are obtained from the motion vector predictor and the differencemotion vector. Similar to the merge mode, in the AMVP mode, an AMVPcandidate list including a motion vector obtained from a neighboringblock of the current block is obtained. Also, the AMVP candidate listmay be determined according to different schemes according to whetherthe motion vector is extractable from the right block.

When the motion vector is not obtainable from the right block, theprediction method determiner 1630 may determine a first motion vectorpredictor candidate from the left block including the reference sample2604 and the lower left block including the reference sample 2606. Then,the prediction method determiner 1630 may determine a second motionvector predictor candidate from the upper block including the referencesample 2608, the upper right block including the reference sample 2610,and the upper left block including the reference sample 2602. Then, theprediction method determiner 1630 may determine the motion vectorpredictor from the AMVP candidate list including the first motion vectorpredictor candidate and the second motion vector predictor candidate.

When the motion vector is obtainable from the right block, theprediction method determiner 1630 may determine the first motion vectorpredictor candidate from the right block including the reference sample2614 and the lower right block including the reference sample 2616.Then, the prediction method determiner 1630 may determine the secondmotion vector predictor candidate from the upper block including thereference sample 2608, the upper right block including the referencesample 2610, and the upper left block including the reference sample2602. Then, the prediction method determiner 1630 may determine a thirdmotion vector predictor candidate from the left block including thereference sample 2604 and the lower left block including the referencesample 2606. When the motion vector is not extractable from the leftblock and the lower left block, a motion vector obtained by scaling thesecond motion vector predictor candidate may be determined as the thirdmotion vector predictor candidate. Then, the prediction methoddeterminer 1630 may determine the motion vector predictor from the AMVPcandidate list including the first motion vector predictor candidate,the second motion vector predictor candidate, and the third motionvector predictor candidate.

The merge mode list of the merge mode may include a motion vectorcandidate obtained from a temporally adjacent block. Equally, the AMVPcandidate list of the AMVP mode may include the motion vector candidatepredicted from the temporally adjacent block.

In inter prediction, left neighboring blocks are not limited toparticular locations such as the left block, the upper left block, thelower left block, or the like, right neighboring blocks are not limitedto particular locations such as the right block, the lower right block,the upper right block, or the like, and upper neighboring blocks are notlimited to particular locations such as the up side, the upper leftside, the upper right side, or the like. Also, in a skip mode, the mergemode, and the AMVP mode, a candidate list may be differently consistedand the number of allowable candidate lists may vary according towhether the left block and the right block have been decoded.

With reference to FIG. 27 , a prediction method using a right block of acurrent block in an OBMC mode will now be described. In the OBMC mode,with respect to samples at a boundary of the current block, a pluralityof prediction values are obtained by using a motion vector of thecurrent block and a motion vector of a block adjacent to the currentblock. Then, a final prediction value of a current sample is obtained byweight-averaging the plurality of prediction values. Inweight-averaging, a weight to a current motion vector is generallygreater than a weight to an adjacent motion vector.

For example, with respect to a sample 2702 located at a left boundary ofa current block 2700, a current motion vector of the current block 2700is obtained, and an adjacent motion vector is obtained from a left blockof the current block 2700, the left block including a left sample 2704of the sample 2702. Then, a final prediction value of the sample 2702 isdetermined by weight-averaging a prediction value obtained from thecurrent motion vector, and a prediction value obtained from the adjacentmotion vector.

For a sample 2706 that is adjacent to both a left boundary and a topboundary of the current block 2700, a current motion vector of thecurrent block 2700 is obtained, a first adjacent motion vector isobtained from a left block of the current block 2700, the left blockincluding a left sample 2708 of the sample 2706, and a second adjacentmotion vector is obtained from an upper block of the current block 2700,the upper block including an upper sample 2710 of the sample 2706. Then,a final prediction value of the sample 2706 is determined byweight-averaging a prediction value obtained from the current motionvector, a prediction value obtained from the first adjacent motionvector, and a prediction value obtained from the second adjacent motionvector.

When a current block is decoded according to an encoding order accordingto a raster scan, a right block is not decoded and thus the predictionmethod according to the OBMC mode is not applied to a sample 2712.However, when a right block of the current block 2700 has been decoded,the prediction method according to the OBMC mode may be applied tosamples located at a right boundary of the current block 2700.

For example, when the right block of the current block 2700 has beendecoded, a current motion vector of the current block 2700 is obtainedwith respect to the sample 2712 located at the right boundary of thecurrent block 2700, and an adjacent motion vector is obtained from theright block of the current block 2700, the right block including a rightsample 2714 of the sample 2712. Then, a final prediction value of thesample 2712 is determined by weight-averaging a prediction valueobtained from the current motion vector and a prediction value obtainedfrom the adjacent motion vector.

With reference to FIGS. 28A to 28C, a prediction method using a rightblock of a current block in a sub-block MVP mode will now be described.The sub-block MVP mode is an inter prediction mode in which a block issplit into sub-blocks, and a motion vector predictor is determined foreach of the sub-blocks.

With reference to FIG. 28A, a method of predicting a current blockaccording to the sub-block MVP mode, when the right block has not beendecoded nor inter-predicted, will now be described. Referring to FIG.28A, a right block of a current block 2800 has not been decoded norinter-predicted and thus cannot be used in inter predicting the currentblock 2800. The current block 2800 includes four sub-blocks 2802, 2804,2806, and 2808. For each of the sub-blocks 2802, 2804, 2806, and 2808,two spatial motion vector candidates are obtained from a block includinga left sample and a block including an upper sample of a sub-block.Also, a temporal motion vector is obtained from a block of a referencepicture, the block including a collocated sample of the sub-block. Then,a motion vector of the sub-block is determined by averaging the twospatial motion vector candidates and the temporal motion vector.

For example, with respect to a sub-block 2802, a first spatial motionvector may be obtained from a block including a left sample 2810 of thesub-block 2802. When the block including the left sample 2810 has notbeen encoded nor inter-predicted, the first spatial motion vector may beobtained by referring to a block including samples located below theleft sample 2810.

A second spatial motion vector may be obtained from a block including anupper sample 2812 of the sub-block 2802. When the block including theupper sample 2812 has not been encoded nor inter-predicted, the secondspatial motion vector may be obtained by referring to a block includingsamples located in the right of the upper sample 2812.

A temporal motion vector may be obtained by referring to a collocatedblock of the sub-block 2802 from the reference picture referred to for acurrent picture. Alternatively, temporal motion vectors of thesub-blocks 2802, 2804, 2806, and 2808 may be determined by referring toa collocated block of the sub-block 2802.

Finally, a motion vector of the sub-block 2802 is obtained by averagingthe first spatial motion vector, the second spatial motion vector, andthe temporal motion vector. For other sub-blocks 2804, 2806, and 2808,motion vectors are determined by using the same method.

With reference to FIG. 28B, a method of predicting a current blockaccording to a sub-block MVP mode, when both a left block and a rightblock of a current block have been decoded according to an interprediction mode, will now be described. Referring to FIG. 28B, a motionvector determining method according to FIG. 28A may be applied tosub-blocks 2822 and 2826 in the left of a current block 2820. However,with respect to sub-blocks 2824 and 2828 in the right of the currentblock 2820, a third spatial motion vector may be additionally obtainedfrom a block including a right sample of a sub-block so as to obtain amotion vector of the sub-block.

For example, a third spatial motion vector may be obtained from a blockincluding a right sample 2830 of the sub-block 2824. When the blockincluding the right sample 2830 has not been encoded nor interpredicted, the third spatial motion vector may be obtained by referringto a block including samples located below the right sample 2830. Then,a first spatial motion vector, a second spatial motion vector, and atemporal motion vector of the sub-block 2824 may be obtained in a mannerdescribed with reference to FIG. 28A.

Finally, a motion vector of the sub-block 2824 may be obtained byaveraging the first spatial motion vector, the second spatial motionvector, the third spatial motion vector, and the temporal motion vectorof the sub-block 2824. A motion vector of the sub-block 2828 may bedetermined by using the same method.

With reference to FIG. 28C, a method of predicting a current blockaccording to a sub-block MVP mode, when a left block has not beendecoded nor inter predicted, will now be described. Referring to FIG.28C, a left block of a current block 2840 has not been decoded nor interpredicted and thus cannot be used in inter predicting the current block2840. The current block 2840 includes four sub-blocks 2842, 2844, 2846,and 2848. For each of the sub-blocks 2842, 2844, 2846, and 2848, twospatial motion vector candidates are obtained from a block including aright sample and a block including an upper sample of a sub-block. Also,a temporal motion vector is obtained from a block of a referencepicture, the block including a collocated sample of the sub-block. Then,a motion vector of the sub-block is determined by averaging the twospatial motion vector candidates and the temporal motion vector.

For example, for the sub-block 2844, a first spatial motion vector maybe obtained from a block including a right sample 2850 of the sub-block2844. When the block including the right sample 2850 has not beenencoded nor inter-predicted, the first spatial motion vector may beobtained by referring to a block including samples located below theright sample 2850.

A second spatial motion vector may be obtained from a block including anupper sample 2852 of the sub-block 2844. When the block including theupper sample 2852 has not been encoded nor inter-predicted, the secondspatial motion vector may be obtained by referring to a block includingsamples located to the left of the upper sample 2852.

A temporal motion vector may be obtained by referring to a collocatedblock of the sub-block 2844 from the reference picture referred to for acurrent picture. Alternatively, temporal motion vectors of thesub-blocks 2842, 2844, 2846, and 2848 may be determined by referring toa collocated block of the sub-block 2848.

Finally, a motion vector of the sub-block 2844 may be obtained byaveraging the first spatial motion vector, the second spatial motionvector, and the temporal motion vector. Motion vectors of othersub-blocks 2842, 2846, and 2848 are determined by using the same method.

With reference to FIGS. 29A and 29B, a prediction method using a rightblock of a current block in an affine MC prediction mode will now bedescribed. In an image, magnification or reduction of an object,rotation, a perspective, and other irregular operations occur. Toexactly predict motion of the object, the affine MC prediction modeusing affine transformation may be used.

The affine transformation indicates transformation on two affine spacesthat preserve collinear points. Here, an affine space is a geometricstructure obtained by generalizing a Euclidean space, and in the affinespace, attributes with respect to measurement of distances and anglesare not maintained, but only collinearity between points, collateralityof lines, and length ratios between points on a same line aremaintained. That is, according to affine transformation, lines andcollaterality of the lines are preserved, and directions and angles ofthe lines, distances between the lines, and areas are not preserved.Therefore, when the object is magnified and reduced, or is rotated, anarea including the object in the image may be exactly predictedaccording to the affine MC prediction mode.

With reference to FIG. 29A, the affine transformation will now bebriefly described. Four vertexes 2902, 2904, 2906, and 2908 of a block2900 respectively correspond to motion vectors 2912, 2914, 2916, and2918. The block 2900 is affine transformed based on the motion vectors2912, 2914, 2916, and 2918, and thus an affine transformation block 2910is generated. Samples located in the block 2900 may match samples of theaffine transformation block 2910.

For example, a sample 2904 that is obtained by affine transforming asample 2922 located at a line connecting the vertex 2906 and a sample2920 located at the top of the block 2900 is located at a lineconnecting a sample 2926 of the affine transformation block 2910 and asample 2928 of the affine transformation block 2910, the sample 2926being indicated by the motion vector 2916 of the vertex 2906 and thesample 2928 being indicated by the motion vector 2930 of the sample2920.a location of the affine transformed sample 2924 may be determinedbased on the motion vector 2930 obtained by linear interpolating themotion vectors 2912, 2914, 2916, and 2918 according to a location of thesample 2922. Equally, other samples in the block 2900 may be affinetransformed and then may be matched with samples of the affinetransformation block 2910. As described with reference to FIG. 29A, allsamples in a block may be inter predicted by using a motion vectorgenerated for affine transformation.

With reference to FIG. 29B, blocks that are referred to in an affine MCprediction mode will now be described.

The affine MC prediction mode includes an affine merge mode and anaffine AMVP mode. In the affine merge mode, blocks that are from amongneighboring blocks of a current block and are predicted according to theaffine MC prediction mode are determined as candidate blocks. Then,information for affine MC is obtained from a block selected from thecandidate blocks. In the affine AMVP mode, at least two motion vectorpredictors that are used in affine transformation are determined fromthe neighboring blocks of the current block. Then, the current block ispredicted by using the motion vector predictor, and a difference motionvector and reference picture information that are included in abitstream. In detail, prediction methods according to the affine mergemode and the affine AMVP mode will now be described.

When the affine merge mode is applied to a current block 2950, candidateblocks are determined from neighboring blocks predicted according to theaffine MC prediction mode of the current block 2950. When a change inencoding orders of blocks is not allowed, it is checked whether each ofneighboring blocks has been predicted according to the affine MCprediction mode, in order of a neighboring block including a left block2960 of the current block 2950, a neighboring block including an uppersample 2964, a neighboring block including an upper right sample 2966, aneighboring block including a lower left sample 2968, and a neighboringblock including an upper left sample 2972. Then, the neighboring blockspredicted according to the affine MC prediction mode are included in anaffine merge list according to the order.

When a change in encoding orders of blocks is allowed, it is checkedwhether each of neighboring blocks has been predicted according to theaffine MC prediction mode, in order of the neighboring block includingthe left block 2960 of the current block 2950, a neighboring blockincluding a right block 2962, the neighboring block including the uppersample 2964, the neighboring block including the upper right sample2966, the neighboring block including the lower left sample 2968, aneighboring block including a lower right sample 2970, and theneighboring block including the upper left sample 2972. Equally, theneighboring blocks predicted according to the affine MC prediction modeare included in an affine merge list according to the order. Therefore,the right block and the lower right block of the current block 2950 maybe included in the affine merge list.

Motion vectors and reference picture information for affinetransformation of the current block 2950 are obtained from a block thatis from among the candidate blocks of the affine merge list and isindicated by an affine merge flag obtained from a bitstream. Then, thecurrent block 2950 is predicted based on the motion vectors and thereference picture information.

When the affine AMVP mode is applied to the current block 2950, at leasttwo motion vector predictors are determined from neighboring blocks ofthe current block 2950. According to an embodiment, three motion vectorpredictors may be determined. For example, a first motion vectorpredictor is obtained from a neighboring block located adjacent to anupper left vertex 2952 of the current block 2950. The first motionvector predictor may be obtained from a neighboring block including asample 2972 located in the upper left of the upper left vertex 2952, aneighboring block including a sample 2974 located in the left, and aneighboring block including a sample 2976 located above.

A second motion vector predictor is obtained from a neighboring blocklocated adjacent to an upper right vertex 2954 of the current block2950. The second motion vector predictor may be obtained from aneighboring block including a sample 2964 located above the upper rightvertex 2954, and a neighboring block including a sample 2966 located inthe upper right. When a change in encoding orders of blocks is allowed,the second motion vector predictor may be obtained from a neighboringblock including a sample 2978 located in the right of the upper rightvertex 2954.

A third motion vector predictor is obtained from a neighboring blocklocated adjacent to a lower left vertex 2956 of the current block 2950.The third motion vector predictor may be obtained from a neighboringblock including a sample 2960 located in the left of the lower leftvertex 2956, and a neighboring block including a sample 2968 located inthe lower left. When a change in encoding orders of blocks is allowed,and a left block of the current block 2950 has not been encoded whereasa right block has been first encoded, the third motion vector predictoris obtained from a neighboring block located adjacent to a lower rightvertex 2958 of the current block 2950. Therefore, the third motionvector predictor may be obtained from a neighboring block including asample 2962 located in the right of the lower right vertex 2958, and aneighboring block including a sample 2970 located in the lower right.

Alternatively, when the right block has been first encoded, the thirdmotion vector predictor may be changed, based on a motion vectorprediction value of the lower left vertex 2956 which is derived by usingthe first motion vector predictor, the second motion vector predictor,and the third motion vector predictor which are obtained from the upperleft vertex 2952, the upper right vertex 2954, and the lower left vertex2956. Then, the changed third motion vector predictor may be used inpredicting a current block.

After the three motion vector predictors are obtained, three motionvectors used in affine transformation are determined based on thereference picture information obtained from the bitstream and threedifference motion vectors. Then, the current block 2950 is predictedaccording to the three motion vectors.

As described above, a plurality of pieces of information necessary forthe affine MC prediction mode may be obtained from a neighboring blocklocated in the right of a current block.

With reference to FIGS. 30A and 30B, a prediction method using a rightblock of a current block in an FRUC mode will now be described. FIG. 30Adescribes a bilateral matching FRUC mode, and FIG. 30B describes atemplate matching FRUC mode.

The FRUC mode is an inter prediction mode based on a frame rate increasetransformation technique. The FRUC mode includes the bilateral matchingFRUC mode and the template matching FRUC mode that are commonlycharacterized in determining a motion vector of a current block by usinga merge candidate list of the current block.

The bilateral matching FRUC mode is an inter prediction mode in which amotion vector of a current block is searched for, assuming that there isa continuity in motions of sequential pictures. According to thebilateral matching FRUC mode, the prediction method determiner 1630obtains a plurality of motion vector candidates from a merge candidatelist. Then, the prediction method determiner 1630 obtains a referenceblock pair from each of the motion vector candidates. Also, theprediction method determiner 1630 compares matching accuracies ofreference block pairs, and determines a motion vector candidate havinghighest matching accuracy as a motion vector predictor of the currentblock. The prediction method determiner 1630 scans a peripheral area ofa point indicated by the motion vector predictor and thus determines amotion vector having machining accuracy that is more accurate than themotion vector predictor. Finally, the prediction method determiner 1630determines the current block according to the motion vector.

Referring to FIG. 30A, a motion vector candidate is obtained from amerge candidate list with respect to a current block 3002 located in acurrent picture 3000. For a first reference picture 3010 that istemporally behind the current picture 3000, a first matching motionvector 3014 is determined based on the motion vector candidate. Then,for a second reference picture 3020 that is temporally ahead the currentpicture 3000, a second matching motion vector 3024 is determined basedon the motion vector candidate.

The first matching motion vector 3014 and the second matching motionvector 3024 are proportional to a temporal distance between the currentpicture 3000 and the first reference picture 3010 and a temporaldistance between the current picture 3000 and the second referencepicture 3020. Therefore, a first reference block 3012, the currentpicture 3000, and a second reference block 3022 are located at a samemotion tracking path. Therefore, assuming that there is continuity inmotions of the first reference block 3012, the current picture 3000, andthe second reference block 3022, matching accuracy of a first referenceblock 3012 and a second reference block 3022 is calculated.

With respect to all motion vector candidates of the merge candidatelist, the aforementioned matching accuracy calculation process isperformed. A motion vector candidate having highest matching accuracy isdetermined as a motion vector predictor of the current block.

Finally, with respect to the first reference picture 3010 and the secondreference picture 3020, a peripheral area of a point indicated by themotion vector predictor is scanned to search for a motion vector havinghigher matching accuracy among reference blocks, and the current block3002 is predicted according to the motion vector.

Because the merge candidate list is used in the bilateral matching FRUCmode, when a right block 3004 of the current block 3002 has beendecoded, a merge candidate list including a motion vector of the rightblock 3004 may be used according to the method described with referenceto FIG. 26 .

The template matching FRUC mode is an inter prediction mode in which atemplate of a current block is compared with a template of a referencepicture corresponded due to a motion vector candidate of a mergecandidate list, and a motion vector of the current block is searched fordue to matching accuracy between the templates. According to thetemplate matching FRUC mode, left and upper areas of the current blockmay be determined as the template of the current block. When a rightblock of the current block has been decoded, a right area of the currentblock may also be determined as the template of the current block.

Referring to FIG. 30B, a motion vector candidate is obtained from amerge candidate list with respect to a current block 3052 located in acurrent picture 3050. Then, a matching motion vector 3062 is determinedfrom the motion vector candidate, according to a temporal distancebetween the current picture 3050 and a reference picture 3060.

According to which neighboring blocks from among neighboring blocks ofthe current block 3052 have been decoded, a current block template 3054is determined. Referring to FIG. 30B, because a right block of thecurrent block 3052 has been determined, the current block template 3054includes a right area of the current block 3050.

A reference block template 3064 is obtained from a point of the currentblock template 3054, the point being indicated by the matching motionvector 3062. Then, template matching accuracy is calculated by comparingthe current block template 3054 with the reference block template 3064.

Template matching accuracy is calculated for each motion vectorcandidate. Then, a motion vector candidate having most accurate templatematching accuracy is determined as a motion vector predictor of thecurrent block 3050.

Finally, a peripheral area of a point of the reference picture 3060indicated by the motion vector predictor is scanned to search for amotion vector having higher template matching accuracy, and the currentblock 3052 is predicted according to the motion vector.

As in the bilateral matching FRUC mode, the merge candidate list is alsoused in the template matching FRUC mode, and thus, when the right block3004 of the current block 3002 has been decoded, a merge candidate listincluding a motion vector of the right block 3004 may be used accordingto the method described with reference to FIG. 26 .

The block decoder 1640 may predict a current block according to aprediction method determined by the prediction method determiner 1630,and may decode the current block, based on a result of the predictionwith respect to the current block.

When split information does not indicate that the current block is to besplit, the block decoder 1640 may obtain, from a bitstream, a finalblock flag indicating whether the current block is a last block of anencoding tree block including the current block.

When the final block flag indicates that the current block is the lastblock of the encoding tree block, the block decoder 1640 may enddecoding of the encoding tree block after the current block is decoded.After the current block is decoded, a next encoding tree block may bedecoded by the video decoding device 1600. As in the encoding tree blockincluding the current block, the block splitter 1610, the encoding orderdeterminer 1620, the prediction method determiner 1630, and the blockdecoder 1640 that are included in the video decoding device 1600 mayperform split of a block, determination of an encoding order, anddecoding of a final split block on the next encoding tree block.

Also, the block decoder 1640 may not obtain the final block flag fromthe bitstream but may determine whether other blocks except for thecurrent block from among blocks included in the encoding tree block havebeen decoded, and then may determine whether the current block is thelast block of the encoding tree block.

The block decoder 1640 may entropy decode a syntax element according toa context of a neighboring block, the syntax element being obtained fromthe bitstream. Therefore, the syntax element may be entropy decoded, inconsideration of encoding information about the right block of thecurrent block.

For example, a skip flag indicating whether the current block has beenencoded according to a skip mode may be entropy encoded according to acontext of neighboring blocks of the current block. Therefore, the skipflag may be entropy encoded, in consideration of encoding informationabout the right block of the current block. Therefore, the block decoder1640 may entropy encode the skip flag, in consideration of the encodinginformation about the right block of the current block.

Equally, split information indicating whether the current block is to besplit into lower blocks, split shape information indicating to whichshape the current block is to be split, an FRUC mode flag indicatingwhether the current block is to be predicted according to the FRUC mode,FRUC mode information indicating which FRUC mode is to be applied to thecurrent block when the current block is to be predicted according to theFRUC mode, an affine mode flag indicating whether the current block isto be predicted according to the affine mode, a motion vector minimumunit flag indicating a minimum unit of the motion vector of the currentblock, or the like may be entropy decoded according to encodinginformation about neighboring blocks including the right block of thecurrent block.

The block decoder 1640 may inverse quantize and inverse transformresidual data obtained from the bitstream. Then, the block decoder 1640may reconstruct the current block by using the inverse quantized andinverse transformed residual data, and the prediction result about thecurrent block.

FIG. 31 illustrates a video decoding method 3100 according to anembodiment involving splitting a current block and determining anencoding order of split lower blocks.

In operation 3110, split information indicating whether a current blockis to be split is obtained from a bitstream.

When the split information does not indicate that the current block isto be split, a final block flag indicating whether the current block isa last block of an encoding tree block including the current block isobtained. When the final block flag indicates that the current block isthe last block of the encoding tree block, after the current block isdecoded, decoding of the encoding tree block is ended. When the finalblock flag indicates that the current block is not the last block of theencoding tree block, decoding is performed on a block in a next order ofthe current block.

In operation 3120, when the split information does not indicate that thecurrent block is to be split, the current block is decoded according toencoding information about the current block.

When the current block is not split according to the split informationbut is inter predicted, reference samples to be referred to for thecurrent block are determined according to whether a left block and aright block of the current block have been decoded. Then, the currentblock is predicted and decoded according to the reference samples.

When only the left block of the current block has been decoded, samplesadjacent to the current block in left and upper directions are includedin the reference samples. When only the right block of the current blockhas been decoded, samples adjacent to the current block in right andupper directions are included in the reference samples. When both theleft and right blocks of the current block have been decoded, samplesadjacent to the current block in right, left and upper directions areincluded in the reference samples. When both the left and right blocksof the current block have not been decoded, samples adjacent to thecurrent block in an upper direction are included in the referencesamples.

When the current block is to be intra predicted according to the DCmode, a prediction value of samples included in the current block isdetermined as an average value of sample values of reference samples,and a prediction value of samples that are from among the samples of thecurrent block and are adjacent to the reference samples is filteredaccording to the sample values of the reference samples.

When the current block is intra predicted according to the planar mode,and the right block and an upper block of the current block have beendecoded, a prediction value of the current sample is determined based ona first corner sample, a second corner sample, a first side sample, anda second side sample that are included in the reference samples. Thefirst corner sample is located at a cross point of a row adjacent to thecurrent block including the current sample in an upper direction and acolumn adjacent to the current block in a left direction. The secondcorner sample is located at a cross point of a row adjacent to thecurrent block in a lower direction and a column adjacent to the currentblock in a right direction. The first side sample is located at a crosspoint of a row of the current sample and the column adjacent to thecurrent block in the right direction. The second side sample is locatedat a cross point of the row adjacent to the current block in the upperdirection and a column of the current sample.

When the current block is intra predicted according to a MPI mode, andthe right block and an upper block of the current block have beendecoded, a prediction value of the current sample may be determinedaccording to a weighted average of a sample value of an upper sampleadjacent to the current sample in an up direction and a sample value ofa right sample adjacent to the current sample in a right direction. Inthis regard, prediction with respect to samples included in the currentblock may start from a sample adjacent to an upper right corner of thecurrent block.

Unlike to the aforementioned example, when a current block is to beintra predicted according to a MPI mode, and a left block, a right blockand an upper block of the current block have been decoded, the currentblock may be divided into a left area and a right area, and samples tobe used in prediction with respect to the left area and the right areamay be determined in different manners. According to another embodiment,when all of the left block, the right block and the upper block of thecurrent block have been decoded, there is no division to a left area anda right area but a prediction value obtained from the left block and theupper block and a prediction value obtained from the right block and theupper block may be averaged or weight averaged and thus may be used as afinal prediction value.

When the current sample is located in the left area, a prediction valueof the current sample is determined by weight-averaging a sample valueof the upper sample adjacent to the current sample in the upperdirection and a sample value of the left sample adjacent to the currentsample in the left direction. On the contrary, when the current sampleis located in the right area, a prediction value of the current sampleis determined by weight-averaging a sample value of the upper sampleadjacent to the current sample in the upper direction and a sample valueof the right sample adjacent to the current sample in the rightdirection.

When a current block is to be intra predicted according to the LM chromamode or an MPC mode, and a right block of the current block has beendecoded, the current block may be predicted by referring to luma-chromasample pairs located in a right side of the current block.

When a current block is to be inter predicted, it is checked whether aright block of the current block has been decoded according to interprediction. When the right block of the current block has been decodedaccording to inter prediction, a motion vector of the current block isdetermined by using a motion vector of the right block.

When a current block is inter predicted according to the merge mode,motion vector candidates are obtained from a right block, a lower rightblock, an upper block, an upper left block, a left block, and an upperright block. When a current block is inter predicted according to theAMVP mode, a first motion vector candidate is determined from a rightblock or a lower right block of the current block, and a second motionvector candidate is determined from an upper block, an upper rightblock, or an upper left block of the current block.

When a current block is inter predicted according to the AMVP mode, anda right block of the current block has been decoded prior to the currentblock, a first motion vector candidate may be determined from the rightblock or a lower right block of the current block, and a second motionvector candidate may be determined from an upper block, an upper rightblock, or an upper left block of the current block.

When a current block is inter predicted according to the OBMC mode, anda right block of the current block has been decoded prior to the currentblock, a right adjacent motion vector of the right block of the currentblock is obtained. Then, a sample located at a right boundary of thecurrent block is predicted by using a prediction value according to themotion vector of the current block and a prediction value according tothe right adjacent motion vector.

When a current block is inter predicted according to the sub-block MVPmode, the current bock is split into a plurality of sub-blocks. One ormore spatial motion vectors may be obtained from a right block, a leftblock, and an upper block of the current block. A collocated block ofthe current block is obtained from a reference picture including thecurrent block, and a temporal motion vector is obtained from thecollocated block. A motion vector of the sub-blocks may be determined byaveraging the one or more spatial motion vectors and the temporal motionvector.

When a current block is inter predicted according to the affine mergemode, affine candidate blocks that are predicted according to affinemotion information including a plurality of motion vectors and are fromamong neighboring blocks of the current block are searched for, and anaffine merge list including the affine candidate blocks is generated.Then, an affine candidate block to be used in predicting the currentblock is determined from the affine merge list, according to an affinemerge flag obtained from a bitstream. Finally, the affine motioninformation is obtained from the affine candidate block. The affinemerge list may be determined by using the method described withreference to FIG. 29B.

When a current block is inter predicted according to the affine AMVPmode, a first motion vector predictor is obtained from blocks adjacentto an upper left vertex of the current block. A second motion vectorpredictor is obtained from blocks adjacent to an upper right vertex ofthe current block. When a left block from among the left block and aright block of the current block has been decoded, a third motion vectorpredictor is obtained from blocks adjacent to a lower left vertex of thecurrent block, and when the right block from among the left block andthe right block of the current block has been decoded, a third motionvector predictor is obtained from blocks adjacent to a lower rightvertex of the current block. Then, affine motion information about thecurrent block is obtained according to the first motion vectorpredictor, the second motion vector predictor, and the third motionvector predictor.

When a current block is inter predicted according to the bilateralmatching FRUC mode, and a right block of the current block has beendecoded, a plurality of motion vector candidates are obtained from theright block, a lower right block, an upper block, an upper left block, aleft block, and an upper right block of the current block. A pluralityof reference block pairs are generated by applying the plurality ofmotion vector candidates to at least two reference pictures. Then, areference block pair having high matching accuracy from among theplurality of reference block pairs is selected, and a motion vectorcandidate used in generating the selected reference block pair isdetermined as a motion vector predictor. A motion vector that generatesa reference block pair having matching accuracy that is higher than thatof the motion vector predictor is searched for by scanning peripheralareas of points with respect to the at least two reference pictures, thepoints being indicated by the motion vector predictor. Then, the motionvector is determined as a motion vector of the current block.

When a current block is inter predicted according to the templatematching FRUC mode, and a right block of the current block has beendecoded, a plurality of motion vector candidates are obtained from theright block, a lower right block, an upper block, an upper left block, aleft block, and an upper right block of the current block. Then, atemplate of the current block is obtained from an upper area and a rightarea of the current block. Alternatively, when both the left block andthe right block have been decoded, a plurality of motion vectorcandidates are obtained from the left block, a lower left block, anupper left block, the right block, the lower right block, the upperblock, the upper left block, and the upper right block of the currentblock. Then, the template of the current block is obtained from a leftarea, an upper area, and a right area of the current block. A pluralityof reference blocks are obtained by applying a plurality of motionvectors to at least two reference pictures, and a plurality of referenceblock templates are obtained from upper areas and right areas of theplurality of reference blocks. Then, a motion vector is determined as amotion vector predictor of the current block, the motion vectorcorresponding to a reference block template that is most similar to thetemplate of the current block and is from among the plurality ofreference block templates. Also, a motion vector that generates areference block pair having matching accuracy that is higher than thatof the motion vector predictor is searched for by scanning peripheralareas of points with respect to the at least two reference pictures, thepoints being indicated by the motion vector predictor. Then, the motionvector is determined as a motion vector of the current block.

In operation 3130, when the split information indicates that the currentblock is to be split, the current block is split into at least two lowerblocks, encoding order information indicating an encoding order of thelower blocks of the current block is obtained from the bitstream, adecoding order of the lower blocks is determined according to theencoding order information, and the lower blocks are decoded accordingto the decoding order.

Functions of the video decoding device 1600 which are described withreference to FIG. 16 may be included in the video decoding method 3100.

FIG. 32 illustrates a video encoding device 3200 according to anembodiment involving splitting a current block and determining anencoding order of split lower blocks.

The video encoding device 3200 includes an encoding informationgenerator 3210 and an output unit 3220. In FIG. 32 , the encodinginformation generator 3210 and the output unit 3220 are illustrated asseparate configuring units, but in another embodiment, the encodinginformation generator 3210 and the output unit 3220 may be combined tobe implemented as one configuring unit.

In FIG. 32 , the encoding information generator 3210 and the output unit3220 are illustrated as configuring units included in one device, butdevices performing respective functions of the encoding informationgenerator 3210 and the output unit 3220 are not required to bephysically adjacent to each other. Therefore, in another embodiment, theencoding information generator 3210 and the output unit 3220 may bedispersed.

The encoding information generator 3210 and the output unit 3220 may beimplemented by one processor according to an embodiment. Alternatively,they may be implemented by a plurality of processors according toanother embodiment.

Functions performed by the encoding information generator 3210 and theoutput unit 3220 of FIG. 32 may be performed by the output unit 130 ofFIG. 1A.

The encoding information generator 3210 may split a current block intoat least two lower blocks, and according to a result of the split of thecurrent block, may determine whether to split the current block. Forexample, when coding efficiency by splitting the current block is good,the encoding information generator 3210 may determine to split thecurrent block, and when coding efficiency by not splitting the currentblock is good, the encoding information generator 3210 may determine notto split the current block.

The encoding information generator 3210 may generate split informationindicating whether the current block is to be split. Then, the encodinginformation generator 3210 may determine a split method for the currentblock according to the coding efficiency, and may generate split shapeinformation indicating the split method for the current block.

The encoding information generator 3210 may determine an encoding orderof lower blocks included in the current block, based on codingefficiency according to the encoding order, and may generate encodingorder information indicating the encoding order of the lower blocks.

When the current block is not to be encoded any more, the encodinginformation generator 3210 may determine a prediction mode with respectto the current block. The encoding information generator 3210 maydetermine the prediction mode with respect to the current block,according to coding efficiencies of prediction modes that are applicableto the current block. The prediction modes that are applicable to thecurrent block may include a directional mode, a DC mode, a planar mode,a MPI mode, an LM chroma mode, an MPC mode, a merge mode, an AMVP mode,an OBMC mode, a sub-block MVP mode, an affine merge mode, an affine AMVPmode, a bilateral matching FRUC mode, a template matching FRUC mode, orthe like.

The output unit 3220 outputs a bitstream including encoding informationabout the current block, the encoding information being generated by theencoding information generator 3210. The encoding information about thecurrent block may include split information, split shape information,split order information, prediction mode information, or the like.

FIG. 33 illustrates a video encoding method 3300 according to anembodiment involving splitting a current block and determining anencoding order of split lower blocks.

In operation 3310, a current block is split into at least two lowerblocks.

In operation 3320, according to a result of splitting the current block,split information indicating whether to split the current block isdetermined.

In operation 3330, according to coding efficiency of the current block,an encoding order of the lower blocks of the current block isdetermined, and encoding order information indicating the encoding orderof the lower blocks is generated.

In operation 3340, a bitstream including split information and theencoding order information is output.

Functions of the video encoding device 3200 which are described withreference to FIG. 32 may be included in the video encoding method 3300.

According to the video encoding technique based on coding units having atree structure which is described with reference to FIGS. 1 to 33 ,image data of a spatial domain is encoded in each of the coding unitshaving a tree structure, and decoding is performed on each largestcoding unit according to the video decoding technique based on codingunits having a tree structure so that the image data of the spatialdomain is reconstructed, and by doing so, a picture and a video that isa picture sequence may be reconstructed. The reconstructed video may bereproduced by a reproducing apparatus, may be stored in a storagemedium, or may be transmitted through a network.

The embodiments according to the present disclosure may be written ascomputer programs and may be implemented in a general-use digitalcomputer that executes the programs by using a computer-readablerecording medium.

While the best embodiments of the present disclosure have beendescribed, it will be understood by one of ordinary skill in the artthat various replacements, modifications, or changes with respect to thepresent disclosure may be made therein without departing from the spiritand scope as defined by the following claims. That is, the claims willbe construed as including the various replacements, modifications, orchanges with respect to the present disclosure. Therefore, thedescriptions provided in the specification and drawings should beconsidered in a descriptive sense only and not for purposes oflimitation.

The invention claimed is:
 1. A video decoding method comprising:obtaining, from a bitstream, encoding order information indicating anencoding order indicating whether a second coding unit is decoded priorto a first coding unit in a current coding unit, the first coding unitand the second coding unit being split from the current coding unit;obtaining encoding information indicating an intra prediction mode forthe first coding unit among a plurality of intra prediction modes, theplurality of intra prediction modes including directional modes andnon-directional modes; and, when the encoding order informationindicates that the second coding unit is decoded prior to the firstcoding unit and the encoding information indicates a first mode amongthe plurality of intra prediction modes, obtaining a predicted sample inthe first coding unit using a left sample, a bottom-left sample, a rightsample, a bottom-right sample and a upper sample, wherein the rightsample and the bottom-right sample are located in the second coding unitadjacent to the first coding unit, the left sample, the bottom-leftsample and the upper sample are located in coding units adjacent to thefirst coding unit, the coding units being decoded prior to the firstcoding unit.
 2. A video encoding method comprising: generating encodingorder information indicating an encoding order indicating whether asecond coding unit is decoded prior to a first coding unit in a currentcoding unit, the first coding unit and the second coding unit beingsplit from the current coding unit; determining encoding informationindicating an intra prediction mode for the first coding unit among aplurality of intra prediction modes, the plurality of intra predictionmodes including directional modes and non-directional modes; and, whenthe second coding unit is decoded encoded prior to the first coding unitand a first mode among the plurality of intra prediction modes is usedfor prediction on the first coding unit, obtaining a predicted sample inthe first coding unit using a left sample, a bottom-left sample, a rightsample, a bottom-right sample and a upper sample, wherein the rightsample and the bottom-right sample are located in the second coding unitadjacent to the first coding unit, the left sample, the bottom-leftsample and the upper sample are located in coding units adjacent to thefirst coding unit, the coding units being encoded prior to the firstcoding unit.
 3. A video decoding device comprising: an extractorconfigured to obtain, from a bitstream encoding order informationindicating an encoding order indicating whether a second coding unit isdecoded prior to a first coding unit in a current coding unit, the firstcoding unit and the second coding unit being split from the currentcoding unit; and a decoder configured to: obtain encoding informationindicating an intra prediction mode for the first coding unit among aplurality of intra prediction modes, the plurality of intra predictionmodes including directional modes and non-directional modes, and whenthe second coding unit is decoded prior to the first coding unit and afirst mode among a plurality of intra prediction modes is used forprediction on the first coding unit, obtain a predicted sample in thefirst coding unit using a left sample, a bottom-left sample, a rightsample, a bottom-right sample and a upper sample, wherein the rightsample and the bottom-right sample are located in the second coding unitadjacent to the first coding unit, the left sample, the bottom-leftsample and the upper sample are located in coding units adjacent to thefirst coding unit, the coding units being decoded prior to the firstcoding unit.